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ABSTRACT 

This report represents a fir3t step in a voluntary 
cooperative effort among Federal and State governments and education 
associations to: identify the most critical needs for education 
policy information; assess the present capacity of the national 
statistical system to address these needs; and provide broad 
direction for system improvements. Thirty-six specific statistical 
recommendations are presented that are intended to provide a context 
for subsequent investigations of resource needs , feasibility, costs, 
burdens, and benefits associated with implementing the national 
education data agenda and the development of a strategic plan to 
improve the data system. The guide examines the nature and adequacy 
cf national data in four major domains: (1) background and 
demograprics; i education resources; (3) school processes; and (4) 
student outcomes . For each domain, the guide discusses the importance 
of the data for policy purposes, the nature and limitations of 
current collections and reports, potential strategies for 
improvement, and specific data improvement recommendations. This 
guide was developed by using a broad-based consensus building process 
by members of the National Forum on Education Statistics. Six figures 
supplement the text. A 67-item list of references is included. 
(SLD) 
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A Guide To Improving the National Education Data System 



PREAMBLE 

In response to the congressional mandate of the Hawkins-Stafford Education Amendments of 1988 
(P.L. 100-297), the National Forum on Education Statistics has adopted this report--/! Guide To 
Improving the National Education Data System-** a first step in a multistage process designed to 
improve the quality of the national education data system. It represents a voluntary, cooperative 
effort among the Federal Government. States, and education associations to: 

• identify the most critical needs for education policy information, 

• assess the present capacity of the national statistical system to address these needs, and 

• provide broad direction for system improvements based on this analysis. 

The Guide addresses a broad array of data collection and statistical reports pertaining to education 
from Federal and State agencies and other sources. The 36 specific statistical improvement 
recommendations included in the Guide are intended to provide a substantive context for .subsequent 
investigations of resource needs, feasibility, costs, burdens, and benefits associated with implementing 
this agenda for improvement and for developing priorities and plans of action based upon the 
findings. These investigations, in turn. wiV lead to the development of a strategic plan that will: 

• determine implementation pruritics based on the system improvements recommended in 
the Guide . 

• identify the considerations that must be addressed before particular recommendations can 
be implemented. 

• describe the steps that should be taken to address these considerations, and 

• establish timetables for implementation. 

The National Forum expects that those who implement statistical policies in education will find the 
Guide valuable in accomplishing data system improvement. Creating an improved national education 
data system based on a spirit of cooperation and consensus building will result in higner quality data, 
superior policymaking, and. ultimately, a more effective and efficient education system. 



RESOLUTION 



The National Forum on Education Statistics adopts the report of the National Education Statistics 
Agenda Committee-/! Guide To Improving the National Education Data Svstem-as representing both 
(1) long range direction for the improvement of the national statistical system and (2) a substantive 
framework to focus future efforts of the National Forum. The membership will investigate issues of 
resource needs, feasibility, burden, costs, and benefits associated with implementing this improvemen 
agenda and develop priorities and plans of action based upon these findings. 
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PREFACE 



An era of education renewal was launched in the early 19R0S when the U.S. Secretary of Education 
warned that America was a ' nation at risk" due to the unacceptable levels of student academic 
performance. This decade closed with the dramatic convening of a national summit on education by 
President Bush and our Governors and the establishment of national education goals. But how are we 
to know if we are achieving our goals and doing a better job of addressing the education needs of our 
students? Concomitant leadership in creating and maintaining a national education statistical and 
informational infrastructure is needed to till out the blueprint of education reform. 

The National Forum on Education Statistics was created in 1989 to help meet this need. An 
outgrowth of the National Cooperative Education Statistics System provided under Public Law 
100-297, the National Forum is an oiganization of State and Federal agencies and national education 
associations responsible for collecting, reporting, and using national educational information. Their 
mission is to collaboratively pursue improvements in our educati m data system. 

Over the past year, representatives of the National Forum, through the National Education Statistics 
Agenda Committee (NEE AC), have worked hard and productively to examine the current status of 
national education information and to propose a set of thoughtful statistical improvement 
recommendations. 

This first report of the National Forum on Education Statistics—^ Guide To Improving the National 
Education Data System— is intended to provide broad direction regarding the types of education 
information that Federal and State agencies should cooperatively focus on over the next decade. The 
credo for this consensus document is that "Good data help to make good policies." The ultimate 
objective is to put into place an education information base that will provide adequate, timely, useful, 
accurate, and comparable data to education policymakers at all levels. 

Production of this document would not have been possible without the significant contribution of a 
number of individuals who deserve to be recognized. First, primary credit for the quality, balance, 
and thoughtfulness of the final product goes to Martin Orland of the National Center for Education 
Statistics. In the face of much ambiguity and some apprehension about the scope of the project, 
Marty provided the needed content and policy guidance, as well as good judgment, to keep our 
committees on task and productive. 

Second, there was the welcome and superlative leadership provided by key Forum representatives 
who contributed an extraordinary amount of time and quality thinking to the conceptualization and 
explication of this report. They are Joel Bloom (New Jersey). Robert Burns (Oregon). Lynn Corneti 
(Southern Regional Education Board) and James Phelps (Michigan). 

Third, this project owes an intellectual debt of gratitude to a cadre of national expert consultants 
whose critical literature summaries and conceptualization of issues helped frame each of the Guide's 
substantive chapters: Roberto Fernandez, Northwestern University (student and community 
demographics); Andrew Porter, University of Wisconsin (school processes); Margaret Goertz. 
Educational Testing Service (education resources); and Leigh Burstein. UCLA {student outcomes). In 
addition, Rolf Blank (Council of Chief State School Officers). Joan Shoemaker (Connecticut 
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Department of Educ tion), Anne Hafner and Edith McArthur (NCES). and David Stevenson (Office 
of Research, OERI) each made significant and important contributions to the Guide 's development. 

Fourth, Marty and I owe a special thanks to Richard Shavelson (University of California-Santa 
Barbara), who provided overall guidance and wisdom throughout the project, and to John Roller 
whose editorial support has proven invaluable. 

Finally, this effort could not have been launched, or sustained, without the leadership of Emerson 
Elliott, Commissioner of ihe National Center for Education Statistics. His vision of a national 
cooperative statistics system in education launched the National Forum on Education Statistics, and 
his unswerving commitment of staff and resources in support of this Guide has begun to transform 
that vision into reality. 

It is our sincere hope that the Guide will stimulate Federal and State actions to address and imnrove 
the current limitations in our intergovernmental education information systems and serve as a catalyst 
for enhancing the quality of the Nation's education institutions. 



Pascal D Forgione. Jr.. Ph.D, 
Chairperson. National Education 
Statistics Agenda Committee. 
The National Forum on Education 
Statistics, and 

Director, Division of Research . 
Evaluation and Assessment. 
Connecticut State Department oJ 
Education 
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The Hawkins-Stafford Amendment of 1988 
(Public Law 100-297) 
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DEPARTMENT OF EDUCATION 







National Center for Education Statistics/ 
Office of Educational Research and Improvement 







The National Forum on Education Statistics 



Membership (92 members as of October 1990) 

o 68 Voting Members: 

- 56 State Education Agency Members, and 
- 12 Federal Agency Members 

o 24 Associate Members: 

- K» National/State Organization Members; ami 

- 8 Federal Organization Members 
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A Guide To Improving the National Education 

Executive Summary 



Good data help to make good policies! That simple credo embodies 
the rationale for this document-lhe first "product" of the newly created 
National Forum on Education Statistics. Prepared by the National 
Education Statistics Agenda Committee (NESAC) of the National Forum, 
the Guide marks a first step in fulfilling the mandate to develop and 
propose an agenda for improving the Nation's elementary and secondary 
education statistics system in order to meet the needs of education 
policymakers, planners, and practitioners in the 1990s and heyond. 

The Guide examines the strengths and weaknesses of the current 
elementary and secondary education data system and presents 
recommendations for impioving the system's usefulness. Much of what we 
say is not new. In recent years scholars, policymakers, practitioners, and 
others have devoted considerable attention to the question of how to 
improve national education data. 

What is unique, and even revolutionary, about the Guide is that it is 
the product of a broad-based, consensus-building process. l : or the fust 
time, representatives of State and Federal education agencies, as well as of 
organizations with a major interest in education data, have agreed on the 
types of improvements that are most important for enhancing the usefulness 
of the national elementary and secondary education statistical data base. 



Good data help to 
make good policies! 
That simple credo 
embodies the 
rationale for this 
document .... 



The Guide examines 
. . . the current . . . 
education data 
system and presents 
recommendations for 
improving the 
system 's usefulness. 
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Executive Summary 



A useful and 
responsive national 
education data 
system must . . . 
accommodate the 
. . . needs of Us 
various "education 
stakeholders," Thus, 
the Guide offers fan] 
. , . itinerary . . . 
to address important 
policy concerns. 



Despite differences in data needs and diverse constituencies, members of 
the National Education Statistics Agenda Committee have worked 
cooperatively to develop a broad agenda for action. 

A useful and responsive national education data system must, to the 
extent feasible, accommodate the high-priority data needs of its various 
"education stakeholders." Thus, the Guide offers a data improvement 
itinerary for overcoming significant limitations in the ability of the present 
data system to address important policy concerns. The recommendations 
represent destination points that the system can, and eventually should. 



reach. 



However, there is a difference between establishing a statistical 



improvement agenda and implementing that agenda. Proposing an itinerary 
of important statistical improvement destinations, while valuable, is not the 
same as determining how best to reach them or even which improvements 
to address first. 

Taking those steps will require additional research that explicitly 
considers the strengths and weaknesses of specific implementation strategies 
from such perspectives as information quality, cost, burden, and 
compatibility with current activities. Thus, the National Forum's next step 
will be to convene a special task force to develop a plan tor implementing 
the statistical system improvements recommended in this Guide. 
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Key Principles and Precepts 

To guide the National Forum toward the goal of creating a national 
system of high-quality, policy-relevant education statistics, the Forum 
developed the following key principles that define the critical characteristics 
of data which the system should produce. The data should: 

• provide valid measures of the underlying phenomena of interest; 

• provide reliable measures of the underlying phenomena of 

interest; 

• be reported at a level of aggregation consistent with the policy 
questions of interest; and 

• be reported in a timely fashion on a schedule that is consistent 
with decisionmaking calendars. 

The National Forum also developed the following five core precepts 
governing the creation of this statistical improvement Guide: 

1 . to focus on the high-priority information needs of education 
policymakers: 

2. to focus on questions of what and why rather than how; 

3. to focus, initially, on education descriptors and indicators; 

4. to focus on four specific data domains- 
baekground/demographics, education resources, school 
processes, and student outcomes; and 

5. to focus on issues of data validity, reliability, level of 
aggregation, and timeliness in identifying current system 
limitations. 

Organization of the Guide 

The Guide examines the nature and adequacy of national data in the 
four major domains of background/demographics, education resources. 



Executive Summary 



The National Forum 
. . . developed the 
following key 
principles [and] . . . 
core precepts .... 



The Guide examines 
. . . national data in 
the four major 
domains of 
background/demo- 
graphics, education 
resources, school 
processes, and 
student outcomes. 
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Executive Summary 



To be truly useful, a 
national education 
statistics system must 
. . . provide data on 
the demographic or 
background "inputs" 
that are likely to 
affect the condition 
and performance of 
the Nation's schools. 



school processes, and student outcomes. For each domain, the Guide: 

• discusses the potential importance of the data for policy 
purposes, including the particular questions that should be 
informed by such data: 

• discusses the nature and limitations of current national 
collections and reports; 

• discusses potential strategies for improvement; and 

• summarizes specific data improvement recommendations. 



Rationale and Important Recommendations by Data Domain 

The following sections of this summary explain the rationale for 
requesting data in each of the four major domains included in this study and 
list the specific statistical improvement recommendations that grew out of 
the analysis of each data domain, 

I. Student and Community Background Statistics 

To be truly useful, a national education statistics system must go 
beyond collecting data about the education system itself. The statistics 
system must also provide data on the demographic or background "inputs" 
that are likely to affect the condition and performance of the Nation's 
schools. The policy questions concerning demographic statistics have a 
number of important implications for data collection and reporting. 

At the most fundamental level, policymakers must have the 
information they need to discern broad trends and patterns in key 
demographic characteristics of students, families, and school communities. 



xii 



Given the mobility of student populations and the frequent changes in their 
circumstances, data on such characteristics should be collected often and 
reported with regularity. 

In addition, accurate, reliable, and comparable data arc needed to 
allocate resources fairly. When jurisdictions employ idiosyncratic 
definitions of student characteristics such as race, income, and attendance 
that are used in allocating education program funds, the integrity and 
fairness of the programs and their funding systems are compromised. 
Thus, whenever demographic data are used to allocate program funds, it is 
especially important that definitions he consistent and uniformly applied. 

Finally, since demographic data are likely to be related to other 
data in many types of analyses, policymakers should be able to look at 
variables of interest by demographic subgroup, particularly in addressing 
questions of equity. Whether a policy question focuses tin individuals 
(e.g.. Are students receiving instruction from "qualified" teachers'.') o. 
aggregates (e.g.. Are schools and districts employing appropriately 
"qualified" instructors?), it is relevant to ask whether the findings are 
consistent for all racial/ethnic groups and social classes, 

Recommendations. The National forum makes the following 
seven recommendations for improving data colh etion and reporting in the 
domain of student and community background statistics: 



Executive Summary 



Whether a policy 
question focuses on 
individuate . . . or 
aggregates . . . , it is 
relevant to ask 
whether the findings 
are consistent for all 
racial/ethnic groups 
and social classes. 
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Executive Summary 



NCES should develop 
the capacity to collect 
and report data on 
private school student 
background 
characteristics that 
are parallel to those 
being developed for 
the universe of public 
school students. 



1. Using data extracted from State administrative record systems 
on the universe of public sciool student:;, the National Center 
for Education Statistics (NCES) should mnually collect and 
report State- and national-level aggregates on the following 
student background characteristics: 

• Fall membership counts by race/ethnicity by grade; and 

• Fall membership counts by sex by grade. 

2. NCES should annually report State- and national -aggregate 
statistics collected by other agencies on the following student 
subgroups: 

• Handicapped students served, by type of handicap; 

• Free-lunch participants; and 

• Participants in compensatory, bilingual, and vocational 
education programs. 

3. NCES, in cooperation with other Federal and State agencies, 
should work toward the regular collect km and reporting of the 
following State and national student background statistics: 

• Limited-English-prollcieney status: 

• Student handicapping conditions by race; 

• Participation in prekindergarten education programs; 

• Student health status {e.g., nutrition, health-related 
absenteeism, and drug and alcohol use); and 

• Student mobility and migrant status. 

4. The Office of Educational Research and Improvement (OERI) 
should fund special studies investigating the efficacy of using 
free-lunch data as proxies for student socioeconomic status 
(SES) and the costs, benefits, and burdens associated with 
regularly collecting and reporting alternative SES measures. 
These studies should specifically examine issues of validity, 
reliability, and usefulness of free-lunch and alternative 
measures for different types of reporting and analysis as well as 
administrative issues related to the collection and reporting of 
such measures. 

5. NCES should develop the capacity to collect and report data on 
private school student background characteristics that are 
parallel to those being developed for the universe of public 
school students. Data might come from the NCES Private 
School Survey and the Schools and Staffing Survey, and they 
should be reported as national aggregates ami. to the extent 
feasible. State aggregates. 
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6. In reporting measures of education resources, school processes, 
and student outcomes from its sample and universe surveys, 
NCES should attempt, to the extent feasible and appropriate, to 
provide disaggregated data using the following student and 
community background characteristics: 

• Sex; 

• Racial/ethnic-group affiliation; 

• Limited-English-proficieney status; 

• Community wealth; and 

• Family income, 

7. NCES should consider reporting distributional patterns for the 
following student and community background variables in 
unijunction with particular resource, process, and outcome 
measures: 

• Public /private school enrollment; 

• Student employment status; 

• Measures of family background (e.g.. parents' education, 
language spoken in the home); 

• Student mobility: and 

• Student handicapping condition. 



II. Education Resource Statistics 

Education resources include both fiscal resources and human and 
nonhuman resources. States-and school districts within States-have 
varying amounts of money available to them, governmental levels 
providing funds (e.g.. Federal, State, intermediate, and local), and funding 
sources (taxation, aid. and nontax revenues). In recent years, education 
policymakers and the public have shown a growing concern about how- 
education resources are allocated and what the relationship is between 
education spending and student achievement. Such concerns focus on five 
key questions: 

1 . What is the total amount spent on elementary and secondary 
education at the national. State, and local levels? 



Executive Summary 



In recent years, 
education 

policymakers and the 
public have shown a 
growing concern 
about . . . the 
relationship . . . 
between education 
spending and student 
achievement. 
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Executive Summary 



Vie Federal 
Government already 
collects most of the 
data needed to 
address these major 
education resource 
policy questions 
.... Titus, some of 
the recommendations 
. . . would require 
enhancements or 
improvements . . . 
rather than new data 
collections. 



2. What percentage of that amount comes from each source of revenue 
(Federal, State, intermediate, local, and private)? 

3. What do education dollars buy at the national. State, and local 
levels? 

4. How are education resources distributed among the States and 
school districts? 

5. How do States allocate education resources given differences in 
levels of student need, fiscal capacity, and cost? 



The Federal Government already collects must of the data needed to 
address these major education resource policy questions, at least tor 
reporting at the national and State levels of aggregation. The redesign of 
the NCES Common Core of Data (CCD) has resulted in the creation of the 
new "National Public Education Financial Survey." which provides the 
most comprehensive and detailed data on education revenues and 
expenditures that have ever been available. Thus, some of the 
recommendations for this domain would require enhancements or 
improvements in current data collections rather than new collections. 

In other resource areas, much developmental work and examination 
of alternative strategies will be necessary before implementation can 
proceed. For example, economists have developed a variety of techniques 
for adjusting resource costs across States and over time (a major 
improvement recommendation in this domain). Each model has its strengths 
and weaknesses: each is appropriate for some purposes more than others; 
and each carries with it different cost and burden implications. Thus. 



considerable work is still needed before the National Forum can 
recommend implementing specific nationally adjusted education resource 
figures. 

Recommendutions. The National Forum makes the following 12 
recommendations for improving data collection and reporting in the domain 
of education resource statistics: 

1 . The National Center for Education Statistics (NCES) should 
collect and report a set of national- and State-level education 
revenue, expenditure, and human resource measures on an 
annual basis, using data items from the "National Public 
Education Financial Survey" and the Common Core of Data 
(CCD) Nonfiscai Surveys. 

2. NCES should continue to provide training *nd technical 
support to States to "crosswalk" data elements specified by the 
current CCD Financial Survey as well as other assistance 
necessary for meeting the Handbook 2R2 classifications. 

3. NCES and other Federal agencies should investigate the 
feasibility of developing a State-by-State statistical measure to 
adjust education resource data for differences among States 
and to report education resource trends over time in constant 
dollars. 

4. NCES and other Federal agencies should investigate the 
feasibility of developing a State-by-State statistical measure to 
adjust salary data for differences among States and to report 
education salary trends over time in constant dollars. 

5. NCES and other Federal agencies should engage in research 
and development efforts that will enable them to make 
accurate, comparable, and informative international 
comparisons of U.S. education resource commitments with 
those of other industrialized nations. 



Executive Summary 

Considerable work is 
still needed before 
the National Forum 
can recommend 
implementing specific 
nationally adjusted 
education resource 
figures. 



NCES and other 
Federal agencies 
should investigate the 
feasibility of ... a 
. . . statistical 
measure to adjust 
education resource 
data for differences 
among States .... 



6. NCES should continue to collect and report data from the 
CCD aggregated to the State level on an annual basis. 
However, NCES should, over time, develop policies and 
procedures for the regular collection and reporting of district- 
level resource data. In moving toward district-level resource 
collections, NCES should be particularly cognizant of 



Executive Summary 



NCES should make a 
long-term 
commitment to 
establishing a 
program- and 
functionally based 
accounting system. 



NCES should 
regularly report data 
on the number and 
descriptive 
characteristics . . . 
of instructional, 
instructional support, 
and noninstructional 
staff in the Nation's 
schools. 



(1) identifying potential reports that such data 
could generate and (2) the capacity of States to 
provide district-level data. 

7. NCES should expand the annual CCD "State Administrative 
Records Survey ' to include: ( 1 ) an average teacher salary 
measure that takes into account contract, career ladder, and 
other special-incentive pay and (2) a teacher salary measure 
that f akes into account degree status and experience. 

8. NCES should make a long-term commitment to establishing a 
program- and functionally based accounting system. This will 
provide NCES. policy analysts, and other education 
researchers v ith better information about how education funds 
are spent and make it possible to relate program resources n» 
the specific education needs of students. The particular 
program levels to be collected should be determined after 
additional study, taking into account the costs and burdens 
associated with the development of comparable definitions of 
relevant program categories across different locales. 

9. NCES should expand the Federal Government's survey of 
private schools to include resource information. Wherever 
feasible, NCES should report private-school resource data 
from its surveys on a State-by-State basis. 

10. NCES should establish, as a long-term objective, 
the collection of data regarding the status of 
buildings, including the number, age. condition, 
and facility needs of the Nation's schools. 

1 1 . NCES should regularly report data on the number and 
descriptive characteristics (i.e.. age. sex. race; of 
instructional, instructional support, and noninstructional staff 
in the Nation's schools. Such data should be reported at the 
State level to the extent feasible. 

12. NCES should establish, as a Jong-term objective, measures 
that indicate total dollar investments in education personnel. 
These measures should be specific to dif ferent types of staff 
(e.g., teachers, administrators, instructional aides) and include 
both direct compensation expenditures (salaries) and indirect 
compensation (fringe benefits). 
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111. School Process Statistics 

School process measures address questions such as who provides 
classroom instruction? what is being taught (and how well)? and what are 
the characteristics of the teaching and learning environment? It is the view 
of the National Forum that school process measures constitute a necessary 
and important component for monitoring the condition of education; 
informing education policy at the national, State, and local levels; and 
providing better mechanisms tor accountability. 

For the policymaker, there are three purposes for regular collection 
and reporting of school process measures. First, process measures can 
describe instructional practice and. wiih this, the degree to which quality 
education opportunities are available to all students in all schools. 

Second, process measures can monitor reform -the degree to which 
recommended changes in education practice are aetu;"y being 
implemented. Kducation in the United States is periodically subject to 
reform efforts that call for substantial changes in current practice, including 
changes in curriculum emphasis, organizational structure, and teaching 
techniques. Monitoring these reforms requires a regular system of 
indicators. 

Finally, process measures can help to explain discrepancies in 
education performance and point to reasons why student achievement may 
vary across locales and over time. For example, if student outcomes are 
improving more in one State than in another, knowledge of differences in 
curricula, instruction, and school organization can provide policymakers 
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Executive Summary 



[NCESJ should . . . 
report national and 
comparable State- 
level data on student 
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academic and 
vocational secondary 
courses by 
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and other 
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with clues to explain these differences and point them toward promis*-" 
future policy directions. 

We have divided our analysis of school process data into the 
following three interrelated sub-domains that, taken together, comprise the 
context of instructional practice. 

• implemented c«rriai/«m--ineludmg what is actually taught in 
classrooms: content and topic coverage, time and emphasis 
devoted to subject areas, course taking, and the :ontext in 
which instruction occurs; 

• teaching ^na/iVy—including professional preparation, use of 
appropriate instructional strategies, acceptance of responsibiliu 
for student success and failure, and certification in assigned 
subject field: and 

• school environment-including academic emphasis, school size 
and structure, curriculum offerings, discipline, staff 
development, and availability of high-technology equipment 
(e.g.. computers). 

Recommendations. The National Forum makes the following six 

recommendations for improving data collection and reporting in the domain 

of school process statistics: 

1 . The National Center for Education Statistics (NCES) should 
regularly collect and report national and comparable State-level 
data on student enrollment in academic and vocational 
secondary courses by race/ethnicity, sex. and other 
demographic subgroups as feasible and appropriate. To 
accomplish this, NCES must first develop procedures for 
ensuring the collection of broadly comparable data across States 
on secondary -school course offerings. The Office of 
Educational Research and Improvement (OERI) 1 should also 
determine the usefulness of collecting State-level data on time 
allocated to subjects in the elementary grades (such as that 
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currently collected in the Schools and Starting 
Survey ISASS) of NCES). 
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NCES should regularly collect and report data at the national 
level on broad indicators of teacher preparation (e.g., 
certification status, number of courses taken in teaching area, 
major field, and preservice and inservice development and 
training experiences) by specific teaching assignment. Trends 
on these measures should be related directly to changes in the 
size of the teacher work force as well as student enrollment 
patterns (i.e., teacher supply and demand). In addition, NCES 
should investigate the feasibility of regularly collecting and 
reporting comparable State-by-State statistics using such 
measures and of reporting on the numbers of new teachers 
certified via "alternative" routes. 

NCES should regularly collect and report data at the national 
level on student "opportunities to learn" specific instructional 
topics. Work should begin first on the high-priority subjects 
included in the national education goals (English, mathematics, 
science, history, and geography) and ihen proceed to other 
subjects. OERI should develop new measures of the depth and 
breadth of coverage for these topics for possible future 
collection and reporting at the national and State levels. 

NCES should regularly collect and report nationally 
representative data on the school environment including school* 
tevel measures of academic emphasis (e.g., curricular offerings 
and emoHments) and decisionmaking practices. To the extent 
feasible. NCES should relate such data to impoitant 
background characteristics of students attending these schools 
(e.g., sex, race/ethnicity, handicapping condition, 
socioeconomic status) as well as to key demographic 
characteristics of the larger school community. 

In order to measure progress in meeting the national goal of 
"safe, disciplined, and drug-free schools" (goal No. 6 adopted 
by the Nation's Governors and the President). NCES or other 
Federal agencies should regularly collect an port national- 
and State-level data on drug and alcohol use «uid violence in the 
schools, as well as on policies and programs undertaken to 
prevent such occurrences. To develop measures of these, 
NCES should proceed immediately to examine the feasibility of 
augmenting its current sample surveys (e.g., SASS), mounting 
a new survey (e.g., using the Fast Response Survey System), 
or working in concert with other agencies concerned with these 
issues (e.g., Centers for Disease Control, Drug Enforcement 



NCES should 
regularly collect and 
report data . . . on 
. . . teacher 
preparation 
. . . student 
"opportunities to 
learn "... the 
school environment 
. . . [and] progress 
in meeting the 
national goal of 
"safe, disciplined, 
and drug-free 
schools" .... 



Executive Summary 



Agency). To the extent feasible, these data should 
be related to the background characteristics of 
students and their home communities. 



OERI should fund 
special studies to 
improve the 
measurement of 
important school 
processes 



m m • 



The Nation 9 s citizens 
and policymakers 
increasingly demand 
infonnation about 
the resu!ts~the 
autcomes-of 
schooling. 



6. OERI should fund special studies to improve the measurement 
of important school processes including academic emphasis, 
subject-specific instructional strategies, depth and breadth of 
content coverage, the use of new technologies in instructional 
programs (e.g., personal computers), and methods of training 
teachers and assessing their competence. Newly developed 
measures created through such special studies may eventually 
be incorporated into future regular rational collections and 
reports 



IV. Student Outcome Statistics 

In past years, parents, legislators. Governors, and leaders of 
business and industry frequently asked questions such as. "How are our 
education dollars being spent?" Today, the question is more likely to be, 
"What is the result of sp miin^ our education dollar*?" The Nation's 
citizen* and policymakers increasingly demand information about the 
i*esulK-the outcomes-of schooling. 

The types of information sought by policymakers about student 
education outcomes are reflected in the following questions: 



What do our students know? Do the\ know as much as 
students in other States and countries? 

How many of our students complete high school? How many 
drop out? How uo our graduation and dropout rates compare 
with those of other States and the Nation as a whole? 



• What do students do after high school? How many attend 
postseeondary institutions? How many enter the military? 
How many enter the job market? How satisfied are they with 
their schooling experience? 



• Are achievement levels, completion rates, attitudes about 
schooling, and the postsecondary-education enrollment and 
employment status of our students improving, staying the same, 
or declining over time? 

These questions reflect the Nation's growing concern ahout what 
students learn throughout their K-12 education and whether students are 
being prepared tor the transition to postsecondary education, employment, 
and adulthood as responsible and productive citizens. The questions also 
illustrate the need for accurate information that policymakers can use in 
making decisions about allocating new education resources or reallocating 
existing ones; continuing current programs or developing new ones; and 
developing or revising policies, rules, and regulations. 

Because States have the primary responsibility for education, it is 
important that they be able to assess and compare their progress toward 
meeting important nationa* goals such as those established by the Governors 
and the President at the 1989 education summit. 

Valid, comparable student outcome measures w.ii improve public 
understanding of the condition of education and may help mobilize public 
interest in and support for the Nation's schools. Conversely, the 
inappropriate collection and reporting of such measures may result in data 
th; j .t are not truly comparable and that do not reflect how schools are doing 
and what students are achieving. 

We recommend that outcome measures he gathered and regularly 
reported in four distinct areas: student achievement, student participation 
and progression, student status after high school, and student attitudes and 
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Executive Summary 



Comparable and 
uniform student 
achievement 
measures , . . should 
provide State-hy-State 
comparisons of 
knowledge in core 
content areas 
(reading, writing, 
mathematics, 
science, history, and 
geography) in grades 
4, H, and 12 ... . 



aspirations. In addition, all outcome measures should he reported by 
race/ethnicity and sex in order to shed light on disparities in education 
achievement among important subgroups of the population, 

Recommendations. The National Forum makes the following 1 1 
recommendations for improving data collection and reporting in the domain 
of student outcome statistics across the Jour key sub-domains: 

Student Achievement 

1. Comparable and uniform student achievement measures (using 
the State National Assessment of Educational Progress {State- 
NAEPJ/ if proven valid and reliable) should 

provide State-by-State comparisons of knowledge 
in core content areas (reading, writing, 
mathematics, science, history, and geography) in 
grades 4. 8. and 12 at least once every 4 years. 
Knowledge in other subject areas such as literature, 
music, art. computer applications, and civics 
should also be periodically assessed to the extent 
feasible. 

2. Differences in performance among important subgroups of 
students should be examined and reported at the nation.*! a;iu 
State levels. Subgroups should include those traditionally 
associated with sex. race and ethnic origin, economic status, 
and language status. Provision should be made for States, if 
they wish, to analyze the sample of the student achievement 
study in their States so that comparisons could he made among 
education units by significant subgroups. 

3. Trends in student performance over time should be reported for 
all grades and subjects in which the achievement data are 
collected at the national and State levels. However, reporting 
trends over time should not restrict the development and use of 
new assessment forms that tap a broader range of student 
proficiencies than those typically associated with "paper and 
pencil" tests. 
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The Office of Educational Research and Improvement (OERl), 
including the NAEP program, should give priority to research, 
development, and experimentation with new assessment 
techniques that can provide broader and more sophisticated 
measures of student performance. 

State-by-State student achievement measures should include, in 
each administration, a performance assessment component(s). 
OERI should enter into cooperative research and development 
arrangements with State and local large-scale assessment 
programs. 

Student achievement results should be scaled in a way that 
allows comparisons with international achievement measures 
such as those from the International Assessment of Educational 
Progress (IAEP) and the International Association for the 
Evaluation of Educational Achievement (IEA). Comparisons 
with international achievement measures should be 
made on a regular basis in order to monitor 
progress in meeting the recently developed national 
education goal adopted by the Governors and the 
President. 

Information should be collected on courses of study completed 
at the time of national and State student achievement 
assessments so that links might be made between 
courses/curricula completed and assessment results. 

Discussion should continue into possible linkages of specific 
features of the National Assessment of Educational Progress 
(NAEP) and the National Education Longitudinal Study 
(NELS) survey instruments as well as better coordination of the 
two surveys by the National Center for Education Statistics 
(NCES). One possibility is to equate the NELS achievement 
instruments to the NAEP items. 
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Student Participation and Progression 

9. NCES, in cooperation with State departments of education, 
should obtain and periodically report comparable State-by-State 
data on school dropouts and completers by race/ethnicity, sex, 
and other important subgroups. The specific measures 
calculated should include: 

• An annual dropout rate as defined in the NCES Dropout 
Field Test or as modified by the results of the field test; 
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• A synthetic cumulative dropout rate: and 

• A school completion rate incorporating, to the extent 
feasible, the recommendations of the Council of Chief State 
School Officers (CCSSO) School Completion Task Force. 



Student Status After High School 

10. NCES. in cooperation with other Federal agencies and State 
departments of education, should investigate the feasibility of 
obtaining and periodically reporting comparable State-by-State 
data on the following subjects by race/ethnicity, sex. and other 
important subgroups: 

• The percentage of high school graduates who enroll in 
different types of postsecondary institutions within a year of 
graduation; 

• The percentage of high school graduates who enter the 
military within a year of graduation; 

• The percentage of high school graduates who enter the 
civilian labor force within a year of graduation; and 

• The percentage of high school graduates in the civilian 
labor force who are employed /not employed one year after 
graduation. 



Student Attitudes and Aspirations 

1 1 . OERI should fund special studies related to the regular 
collection and reporting of data on student attitudes toward 
education and schooling and their future aspirations. These 
studies should investigate both the technical validity and 
reliability of potential statistics of this type and their perceived 
usefulness for purposes of education policymaking and 
planning. 

Expectations and Future Actions 

The 36 recommendations contained in the Guide provide an 
ambitious but essential initial blueprint for reform of the national 
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elementary and secondary education data collection and reporting system. 
Implementing these improvements would substantially alter the landscape of 
this system. 

It is important to make several points about the potential impact of 
the recommendations. First, many of the recommendations can be 
implemented through enhancements or modifications of existing surveys 
rather than through new data collections. In these cases, implementation is 
likely to be more feasible and less costly than might otherwise be true. The 
tables that accompany this document identify the specific agencies and 
national surveys that may be affected by implementing the recommendations 
contained in the Guide. 

Second, a basic data system infrastructure is being created through 
the National Cooperative Education Statistics System for implementing 
many of the statistical improvements we contemplate. Third, there appears 
to be a reasonable balance of burdens between the States and the Federal 
Government associated with implementing the recommended improvements. 

Finally, although some recommendations can be acted upon 
relatively quickly, others will require considerable time. 

What are our expectations for this document? First and foremost. 

we expect that the Guide will begin a systematic process of national 

reform in education statistics. Specifically, we expect that: 

• all members and associates of the National Forum will commit 
their constituent organizations to investigating the possibility of 
making the improvements necessary to meet the objectives 
outlined in the data improvement recommendations; 



Executive Summary 



Many of the 
[Guide's] 

recommendations can 
be implemented 
through 

enhancements or 
modifications of 
existing surveys 
rather than through 
new data collections. 



We expect that the 
Guide will begin a 
systematic process of 
national reform in 
education statistics. 



9 

ERIC 



33 



XXWI 



Executive Summary 



The National Forum 
will develop a 
strategic plan for 
implementing the 
recommendations 
based on the results 
of these discussions. 



• this guide wilt serve as a basis for subsequent interchanges 
among members of the National Forum and relevant 
agency(ies) at the Federal, State, and local levels on strategies 
for implementing these recommendations: and 

• the National Forum will develop a strategic plan for 
implementing the recommendations based on the results of 
these discussions. 

Our expectations for this report are ambitious. We believe that the 

broad-based, consensus-building approach by which the report was 

developed gives credence to its recommendations. We anticipate that those 

who develop and implement education statistical policies will find this 

improvement agenda useful and will take the agenda seriously. We hope 

they believe, as we do, that creating a national education data system based 

on a spirit of cooperation and consensus building will result in the highest 

quality data, superior policymaking, and. ultimately, a more effective and 

efficient education system. 
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Potential Data Development Implications of National Forum on Education Statistics Guide Recommendations: 

Student and Community Background Statistics 

(Appearing on Pages 105-107 of National Agenda Guide) 



Data Implications for: 



Recommends km* 
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of Data 
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Staffing 
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of Educational 
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le* would have to commit additional eiTort /resources to implement the recommendations* 



ACYF ~ Adm migration for Children. Youth, and Families, Department 

of Health and Human Services 
Bl S - Bureau of Labor Statistics, Department of Labor 

CDC - Centers for Disease Control, Department of Health and Human Services 
Census - Bureau of the Census, Department of Commerce 
FNS - Food and Nutrition Service, Department of Agriculture 
NCES ~ National Center for Education Statistics 



OBEML A - Office of Bilingual Education and Minority Language Atiatr> 
OCR * Office for Civil Rights 

OESE = Office of Elementary and Secondary Education 

OME » Office of Migrant Education 

OPBE » Office of Planning. Budget, and Evaluation 

OSERS = Office of Special Education and Rehabilitative Services 

OVAE « Office of Vocational and Adult Educatkm 
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Potential Data Development Implications of National Forum on Education Statistics Guide Recommendations: 

Student and Community Background Statistics (continued) 
(Appearing on Pages 105-107 of National Agenda Guide) 



Data Implications for: 1 1 
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♦Where indicated, States would have t»> Ciimmit additional effort/resources to implement the recommendations. 

BLS = Bureau of Labor Statistics, Department of Labor OME = Offke of Migrant Education 

Census - Bureau of the Census. Department of Commerce OPBE = Office of Planning, Budget, and Evaluation 

NCES National Center lor Education Statistics OSERS -■ Office of Special Education and Rehabilitative Services 



Potential Data Development Implications of National Forum on Education Statistics Guide Recommendations: 

Education Resource Statistics 

(Appearing mi Pages J 08- 110 of National Agenda Guide) 



Data implications for: 
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Potential Data Development Implications of National Forum on Education Statistics Guide Recommendations: 

Education Resource Statistics (continued) 

(Appearing on Pages 108-1 10 of National Agenda Guide) 



Data Implications for: 
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Potential Data Development Implications of National Fonim on Education Statistics Guide Recommendations: 

School Process Statistics 
(Appearing oa Pages 111-112 of National Agenda Guide) 



Data Implications for: 
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Student Outcome Statistics 
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Potential Data Development Implications of National Forum on Education Statistics Guide Recommendations: 

Student Outcome Statistics (continued) 

(Appearing on Pages 1 13-1 15 of National Agenda Guide) 
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Chapter 1 



Introduction 

Good data help to make good policies! That simple credo embodies the rationale for this 
document-the first "product" of the newly created National Forum on Education Statistics. Prepared 
by the Forum's National Education Statistics Agenda Committee (NESAC). the Guide To Improving 
the National Education Data System is the first step in fulfilling our mandate to develop and propose, 
cooperatively, an agenda for improving our national statistical system in education in order to better 
meet the needs of education policymakers and planners in this decade and beyond. 

The purpose of the Guide is to present arguments and recommendations for improving the 
usefulness of data derived from national education statistical collections and reports. Much of the 
Guide 's content is not new. As recognition of the need to improve national education performance 
has grown in recent years, scholars, policymakers, and practitioners have devoted considerable 
attention to the question of how to improve national data systems in order to provide a better 
assessment of current education conditions, identify areas of concern, and monitor education progress. 
Many of their findings and views have been included in the Guide, and we have engaged some of 
these individuals as special consultants to this project. 

What is unique, and even revolutionary, about this effort is that it is the product of a broad- 
based, consensus-building process among key officials in the education policy community. For the 
first time, representatives of State and Federal education agencies and of organizations with a major 
interest in education data have, after due deliberation, agreed on the most important types of 
improvements that are needed to enhance the usefulness of the national statistical data base. Despite 
the differences in our data needs and the diverse constituencies we represent, we have jointly and 
cooperatively developed a broad agenda for action as a first step toward implementing statistical 
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policy reforms. This is a considerable achievement, one that we think lends weight to the 
recommendations offered. 

One can think of the Guide as a "data improvement itinerary" that grew out of our efforts to 
identify significant limitations in the ability of the current system to meet important policy concerns. 
The recommendations for improvement represent the desired "destination points* on the itinerary; that 
is, we have recommended adding important types of information that we believe the system can, and 
eventually should, provide. 

Even though this improvement agenda is relatively long (we make 36 specific 
recommendations for improving the system), it is not a "laundry list" of every piece of data that our 
technology is capable of providing. On the contrary, the lengthy consensus-building process 
undertaken in developing this document was built upon explicit policy information needs expressed 
by national education data users and providers-needs that are not being fully met by the current data 
system. 

The breadth of the recommendations contained in the Guide reflects the diversity of 
constituencies that rely on education data and the different purposes for which they use the data. 
Thus, different constituencies have disparate system improvement concerns. A useful and responsive 
national education data system must, to the extent feasible, accommodate all the high-priority data 
needs of its various "education stakeholder" constituencies. The Guide represents a necessary and 
important first step on the road to a more broadly responsive national education data system. 

At the same time, we recognize the difference between setting and implementing a statistical 
improvement agenda. An itinerary is not a plan. Presenting an agenda of important statistical 
improvement destinations, while valuable, is not the same as determining how best to reach them or 
even which improvements to address first. To do this requires additional research that explicitly 
considers the strengths and weaknesses of specific implementation strategies from such perspectives as 
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information quality, cost, burden, and compatibility with current activities. Thus, we believe that the 
cooperative development of a strategic plan for statistical improvement, based on the Guide, should 
constitute the National Forum's next step in the system improvement process. 

The cooperative decisionmaking model that shaped the Guide and informs other activities of 
the National Forum reflects the spirit of the National Cooperative Education Statistics System 
(Cooperative System) created by the Hawkins-Stafford Education Improvement Amendments of 1988 
(P.L. 100-297). Our committee, and the National Forum, have been given a mandate to implement 
an ongoing cooperative statistics system in education. Composed of Federal and State policymakers 
who collect and report education statistics, as well as representatives of associations and interest 
groups that are among the key providers and consumers of such data, the National Forum attempts to 
foster discussion and consensus on issues concerning statistical policy in education. 

The time is particularly ripe for this agenda-setting enterprise. Policymakers at all levels are 
beginning to define measurable education performance goals and to think seriously about the data 
systems that will be needed to monitor progress toward attaining these goals. However, experience 
has taught us that useful education data systems cannot be created through administrative fiat. 
Developing such systems requires the active participation and support of a diverse group of actors, 
data providers, and users. 

The National Forum's Federal-State membership and cooperative orientation make it the idea! 
arena for discussing how to improve the Nation's education statistics system and for forging the 
broad-based support necessary for such improvements to take place. Thus, we expect the Forum to 
be instrumental in helping to define and implement national statistical policies in education through 
the cooperation and support of its members. 

It is important to emphasize that the efforts of the Cooperative System, the National Forum, 
and this committee will be continuous and e\olving. Effective and efficient data systems must adapt 



to new policy concerns and to the latest advances in research and technology. The Guide should be 
viewed as an initial foray into statistical agenda setting rather than as a finished product, fn the 
future, we expect not only to research and plan implementation strategies based on the items in the 
Guide but also to define new areas for exploration and to revisit old issues based on our 
membership's changing needs and perspectives. 

A. Background 

The Guide has been prepared for the National Forum on Education Statistics by the National 
Education Statistics Agenda Committee (NESAC), one of the Forum's four standing committees. 
Beginning with its first meetings in March 1989, NESAC has defined its mission both within the 
context of the National Forum and in relation to other efforts aimed at improving Federal education 
statistics. The scope of NESAC's role has continued to evolve through subsequent meetings in 1989 
and 1990. 

Several conclusions regarding the nature of this mission require elaboration because they have 
helped shape the orientation, tone, and perspective of the Guide. A conceptual framework describing 
our perceptions of how national education data can be used to inform and improve the policymaking 
process appears in the following section. This framework is followed by a discussion of the key 
principles and precepts that have guided our work. 

B. Consensus and Cooperation in a Decentralized Policy System 

Collections and presentations of national statistical data about education have become 
increasingly prevalent in recent years. In the 1980s, we witnessed the introduction of the U.S. 
Secretary of Education's annual State Education Performance Chan (the "Wall Chart"), which ranks 
and compares States along several dimensions considered relevant to the education enterprise- 



6 



including pupil-teacher ratios, per-pupil expenditures, State reform policies, and achievement on 
college entrance examinations. 

We also saw the creation and expansion of regular national data surveys by the National 
Center for Education Statistics (NCES) covering student and school demographics (the Common Core 
of Data nonfiseal surveys), schooling processes (Schools and Staffing Survey), financial resources and 
expenditures (the Common Core of Data fiscal survey), and student achievement (National 
Assessment of Educational Progress 1NAEPJ ind the National Educational Longitudinal Survey of 
1988 1NELS:88]). 

The Council of Chief Slate School Officers (CCSSO) and the National Governors' Association 
have begun to publish annual reports on State education indicators and reform policies, respectively. 
In addition, the Council has embarked on a major effort to develop comparable indicators of course- 
taking patterns in mathematics and science in each of the 50 States. Finally, international 
comparisons along such dimensions as ed.ie^tion spending and academic achievement have become 
increasingly commonplace as Americans have come to recognize the close relationship between 
education performance and global economic competitiveness. 

Recent commitments by the President and the Nation's Governors to develop national 
performance goals-and measures to determine whether these goals have been attained-indicate that 
the thirst for more and better national education statistics will continue for the foreseeable future. 
One of the principal purposes of these efforts is to inform government policymakers, education 
"stakeholders." and the public about the condition and status of the education system. At the same 
time, the particular needs of these information users are critical to defining the types of data to be 
collected and reported. 
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What policy needs could be addressed by national education statistics? A model of the 
dynamic relationship between potential national statistics and education policy and practice appears in 
figure J. 

As can be seen, national education statistics could potentially provide important information 
for a variety of users and purposes. Federal, State, and local government officials could, for 
example, all use national demographic data in their program funding formulas and for determining 
when initiatives are required to address the special needs of changing student populations. Student 
achievement data could help these same officials-as well as education associations, interest groups, 
and the public-gauge how well students are learning and where improvement efforts should be 
focused. 

There exists a panoply of potential data needs and diverse actors who could be informed by 
national education statistics and who can, in turn, help shape the nature of these statistics. However, 
in a decentralized and fragmented policy environment such as that existing for education, it is logical 
to ask how to ensure that valid and useful national data are collected and reported. 

Cooperation among the different actors and governmental levels is one indispensable 
prerequisite for achieving this objective. Policy officials and data providers in different parts of the 
system need to cooperate and support one another if their own data needs are to be fulfilled. For 
example, Federal policy officials cannot allocate program funds equitably throughout the States and 
school districts without obtaining from these units comparable counts of students and dollars using 
uniformly applied definitions. 

States and localities, in turn, often rely on the Federal Government to orchestrate and 
administer standardized national data collections and reports so that they can accurately compare the 
performance and status of their school systems with those of other States and localities. The 
cooperation and support of members of education associations and interest groups (such as teachers 
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Figure 1 
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and administrators) are also essential for ensuring the availability of timely and accurate statistical 
information. At the same time, these groups are active consumers of these data and, thus, have a 
substantial stake in the nature and quality of the information being provided. 

It also must be recognized that the diverse set of policy officials, data providers, and 
concerned citizens who comprise the constituency for education statistics often have very different 
notions about what types of national data are most important. While State and local school officials 
may perceive relatively little need for international and interstate comparisons of school financing 
arrangements, these officials might be quite interested in data of this type that compare districts within 
their States. Conversely. Federal policymakers are likely to have greater interest in interstate and 
international school finance data than they would in detailed intrastate information. 

For teachers' groups, the most valuable data would probably be on the status of their 
members along such dimensions as background, skill levels, salary, working conditions, attitudes 
toward their profession, and areas of staff shortages/excesses. By contrast, Sta"; policymaking 
organizations such as the Council of Chief State School Officers and the National Governors' 
Association, and the public, are more likely to be interested in a smaller set of broad indicators that 
describe critical characteristics and trends in the teaching profession. 

It is this combination of data interdependency and unique data needs among the different 
participants in the education policy arena that makes the search for consensus on a national statistical 
improvement agenda so compelling. In a decentralized and fragmented policy environment, 
cooperation and compromise among the many data users and providers are essential to meeting the 
priority information needs of all parties effectively and efficiently. 

In suggesting the importance of arriving at a common national statistical improvement agenda, 
we are not assuming a "rationalist" view of the education decisionmaking processes. We do not 
believe, for example, that potential users of education information always (or even frequently) sift 
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through all available data on an issue and then take action based solely on the weight of the evidence. 
We are cognizant of other decisionmaking influences-politics, past practice, and persoral beliefs to 
name but a few. 

However, we feel strongly that in those instances where data are sought by policymakers or 
others, the likelihood that the data will be used effectively and appropriately will be governed by the 
timeliness, usefulness, and quality of the information available. 

This is a unique and opportune moment to improve the Nation's education data system. The 
desire for credible, policy-relevant information on the status and performance nf our education system 
has never been stronger. With this interest has come an increased recognition of the importance of 
having quality data for improving the education system, as well as some of the limitations of currently 
available data. The need fur improved education data has also made education information users 
acutely aware of the need to act jointly and cooperatively in order tor improvements to take place in 
the national data system. 

Thus, the need at this moment is for leadership in creating a national statistical agenda in 
education that represents consensus and cooperation among a diverse set of education constituencies in 
order to produce high-quality, policy-relevant data. The National Forum was created for precisely 
that pu/pose. 

C. Improving the National Education Data System: 
Key Principles and Precepts 

To guide the National Forum toward the objective of creating a national system of high 
quality, policy-relevant education information, the Forum developed the following set of principles 
that define the critical characteristics of data the system produces: 
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1. The data should be used to provide valid measures of the underlying phenomena 
of interest. 

This is not as self-evident as it may sound. The validity of data must be judged in terms of 
how the data are used. Many data collections purport to measure an important phenomenon (such as 
student achievement or education spending levels), but a close examination may reveal a measure that 
is seriously flawed for its intended purposes. Using minimum competency test scores to gauge the 
overall knowledge of students (rather than recognizing that these tests measure only a narrow range of 
minimum proficiencies, at best) is one example of an invalid use of a measure. Another is using 
information from a population "sample" (such as college-bound students in a State) to make inferences 
about the characteristics of a larger group that it does not adequately represent (such as all students in 
a State). While even the best data represent imperfect portrayals of underlying phenomena, care 
should be taken to ensure the maximum correspondence between the goal being measured and the data 
collected to inform it. 

2. The data should provide reliable measures of the underlying phenomena of interest. 
Applying nonstandard and, hence, inconsistent definitions to data across locales poses the 

greatest potential threat to the reliability of national statistical reports. Using inconsistent definitions 
leads to inaccurate national and subnational estimates and compromises the usefulness of comparisons 
across units. Thus, the creation of reliable national (and, where appropriate, international) data 
measures requires the development of standard, uniformly applied definitions of desired data 
elements. 

3. The data should be reported at a level of aggregation consistent with the policy 
questions of interest. 

Some education data users are interested mainly in reports at the national aggregate level, 
especially when their principal concerns are in understanding broad trends over time or making 
international system comparisons. However, many users are equally interested (and occasionally 

12 



58 



more interested) in statistical breakdowns by major geographic (e.g., region. State, school district) 
and demographic (e.g., system size/urbanieity, racial/ethnic group) subunits to address their policy 
needs. Reporting exclusively at high levels of aggregation (such as single national statistics on pupil 
enrollments, per-pupil spending, reading achievement, etc.) can mask important variability within the 
reported unit, resulting in the loss of potentially important information. 

For example, only through the collection and reporting of student achievement data by race in 
the last report of the National Assessment of Educational Progress was it possible to determine that 
recent academic achievement trends of blacks differed substantially from that of the Nation as a 
whole. Sometimes, the use of disaggregated data along more than one dimension (e.g., black 
achievement trends by State or region) may be considered appropriate to address an important policy 
information need. 

4. The data should be reported in a timely fashion on a schedule that is consistent with 
decisionmaking calendars. 

Both the timing and timeliness of national statistical reports are relevant to their potential 
usefulness. The periodicity (annual, twice a decade, etc.) and recency (one-year-old data, five-year- 
old data, etc.) of the data reported should be governed by the information's intended uses as well as 
its likely volatility over time. Reporting enrollment data once every five years, for example, would 
not meet the needs of policymakers wishing to allocate program funds annually based on these counts. 
However, a five-year collection cycle might be considered appropriate for reporting on student 
achievement in particular sub-domains such as science and social studies, especially if scores were nut 
expected to vary greatly over shorter intervals. 
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The Forum has also developed the following set of five core precepts governing the creation 
of this statistical improvement guide: 

1 . To focus on the high-priority information needs of education policymakers; 

2. To focus on questions of what and why rather than how; 

3. To focus initially on education descriptors and indicators; 

4. To focus on specific data domains-background/ demographics, school 
processes, education resources, and student outcomes; and 

5. To focus on issues of data validity, reliability, level of aggregation, and 
timeliness in identifying current system limitations. 

Each precept is elaborated below, 

1, To focus on the high-priority information needs of education policymakers 

As emphasized earlier, this agenda for national data system improvements is driven by the 
questions and needs of those who establish and implement education policies at the Federal, State, and 
local levels. It is assumed that these individuals rely, at least in part, on national statistical data 
collections and reports to shape their policy decisions. The process of developing a statistical action 
agenda begins with identifying critical current and anticipated policy questions that could be addressed 
by national statistics and examining the extent to which the current national statistical system meets 
these needs. The recommendations for data improvement contained in the Guide emanate from this 
analysis. 

The Guide is considerably broader in orientation and scope than other current efforts to 
improve selected elements of the national education data base because it seeks to respond to the 
ir/ormation needs of such a diverse group of data providers and users. There are a number of efforts 
underway to address specific national data concerns, such as measuring progress in meeting the 
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national education goals recently endorsed by the President and the Nation's Governors and 
developing national "indicators" of the condition of our education system. 

While these efforts should be viewed as important parts of the National Forum's statistical 
improvement agenda, they are not synonymous with this agenda. Many, if not most, of the 
recommendations contained in the Guide are concerned with improvements in the data system 
requested by education policymakers that go beyond the scope of current national education goals or 
potential national education indicators. 

2. To focus on questions of what and why rather than how 

The new data agenda focuses primarily on what improvements in current national data 
collections and reports would be most useful to policymakers and why such changes are warranted. 
Yet in making these recommendations for improving the statistical system, we have not ignored issues 
of feasibility and burden. We believe that all the recommendations offered are technically feasible, at 
least in the long run. We also suggest specific implementation strategies and point out areas in which 
considerable developmental work is in order. 

What the Guide does not do is endorse any particular plan of action as the most appropriate 
mechanism for meeting a recommendation objective. To do so would require a more thorough 
investigation of alternative implementation strategies and issues associated with them-ineluding data 
quality, cost, burden, and current system capabilities-than has yet taken place. Once the broad 
improvement agenda has been adopted, we assume that developing plans for its implementation are 
the logical and necessary next steps in the statistical improvement process to be addressed 
subsequently and comprehensively by the National Forum. 

3. To focus initially on education descriptors and indicators 

While the National Forum may make recommendations pertaining to any aspect of national 
statistical policy in education, we believe that the initial focus should be on improving "stand alone" 



descriptive national counts and measures of important elements of the education system and broad 
indicators of the system's status and performance. The illustration in figure 2, taken from a report 
prepared by the Council of Chief State School Officers, describes six distinct purposes of a national- 
State education data system and points out where data may be obtained to meet these purposes. The 
Guide primarily addresses the first three purposes: 

• counting/measuring elements of the education system (e.g.. numbers and 
types of schools* students, dollars spent, achievement test scores); 

• describing system dimensions (e.g., courses taken, achievement levels 
attained); and 

• monitoring important trends and patterns in the other two areas, through 
both cross-sectional examination and over time. 

Subsequent activities of the National Forum may focus on improving the national statistical 

system for other purposes, such as exploring important research questions (e.g.. the relationship 

between education achievement and future economic well-being) and evaluating the effects of special 

programs. 

4. To focus on specific duta domains 

The Guide addresses specific domains or categories of statistical data that broadly reflect the 
important components of our education system. These domains provide information for understanding 
the context, inputs, and outcomes of education. The data domains are; 

• background/demographic data the background characteristics of students 
attending school and the communities where schools are located: 

• education resource data-the financial and human resources available to 
schools; 

• school process data-specifieally curricula content, the quality of instruction, 
and the schooling environment; 

• student outcome data-how well students learn and achieve both in school 
and later in life. 
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Figure 2 

NATIONAL/STATE EDUCATION DATA SYSTEM 

Adapted from a figure developed by the State Education Assessment Center, 

Council of Chief State School Officers 
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Because the Guide is concerned primarily with counting/measuring and describing critical 
aspects of the Nation's education system, its principal objective is to examine the n stand alone" value 
of data within each of the above domains, not relationships among them. Sometimes, however, we 
will note the need to link data across domains to address specific policy concerns. 

In particular, it is clear that regularly collecting and reporting background/demographic data 
has major policy value beyond describii» e the broad character of student populations along such 
dimensions as race, income, and English-language proficiency. This type of data is also important as 
a relational measure for examining how resource levels, schooling processes, and outcomes might 
differ for students from different family and community backgrounds. For this reason, much of the 
discussion of demographic data in the Guide concerns the appropriate breakdowns for collecting and 
reporting school resource, process, and outcome information. 

5. To focus on issues of data validity, reliability, disaggregation, and timeliness 

Finally, the Guide looks at the adequacy of the current national statistical system in education 
from the perspectives of data valiuity, reliability, disaggregation, and timeliness that were noted 
earlier. For this reason, nearly all of the recommendations for improving current statistical 
collections and reports would be expected to have implications for one or more of the following; 

• new national data collections to address important policy concerns with 
appropriate measures; 

• improving the comparability of existing national data reporting across 
important subunits; 

• creating more disaggregated reporting levels to allow for comparisons 
across important geographic and demography subunits; and 

• expanding the frequency of current collection and reporting cycles to 
enhance policy relevance. 
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D. Organization of the Guide 

The following chapters examine the nature and adequacy of national data in the four major 
domains-hackground/demographics, education resources, school processes, and student outcomes. In 
Chapters 2-5, we address the current and potential status of statistical measures in each of these areas 
of the national statistical system. The chapters are structured similarly. In each, we: 

• discuss the potential importance of the data for policy purposes, including 
the particular questions that should be intormed by such data; 

• discuss the nature and limitations of current national collections and reports 
and potential strategies for their improvement; and 

• summarize our specific improvement recommendations. 

In Chapter 6, we provide a complete list of all of the recommendations contained in the 
Guide, and we discuss the National Forum's planned "next steps" in the statistical improvement 
process. 

i 
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Chapter 2 

Student and Community Background Statistics 



A. Rationale and policy questions - p. 23 

B. Current data collections and their limitations - p. 28 

C. Recommendaf tons for improving collection and reporting - p. 34 



This chapter is based on an "idea paper" prepared for the National Forum by Dr. Roberto 
Fernandez of Northwestern University. The draft manuscript for the chapter was written by 
Dr. Martin Orland and Ms. Edith McArthur of the National Center for Education Statistics, 
was prepared in cooperation with the Forum subcommittee on *»udent and community 
background statistics chaired by Dr. Joel Bloom of the New ,? ••« • < Department of Education. 



Student and Community Background Statistics 



To be useful, national education statistics cannot be limited to data about the education system 
alone. The data must also provide information on relevant variables that are likely to affect the status 
and performance of that system; that is. there must be information on the demographic "inputs" to 
schooling. In this chapter we note the policy uses f student and community background data, discuss 
technical issues to be considered in the selection of student and community background measures, 
review current collections of such data, and. finally, recommend specific improvements in these 
national data collections and reports. 

A. Rationale and Important Policy Questions 

There are tour major reasons why it is important to collect national -level demographic la on 
the characteristics of students and their communities, 

• Demographic data make other data more meaningful. Whether policymakers 
are interested in data describing important characteristics of the education system or in 
assessing its performance, including high-quality student and community background 
information is often necessary to provide a meaningful context for the statistic being 
reported. For example, trends in such conditions as sludent/statf ratios and new 
facilities construction become more meaningful and useful when accompanied by 
statistics on enrollment trends. Falling pupil/staff ratios would mean something quite 
different to policymakers in a context of declining, rather than inere....:ng, enrollments 
as would the construction of additional schools. Similarly, in assessing trends in the 
performance of the education system, it is also important to consider trends in relevant 
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input characteristics (such as the socioeconomic status of students) because of their 
known relationships to achievement. 

• Demographic data are needed for allocating funds. Most current Federal and State 
funding formulas incorporate student and/or community demographic data. These data are 
used to allocate funds based on the total number of pupils enrolled in the funded jurisdiction 
(i.e., school, school district. State) as well as on the number with identified "special needs" 
for purposes such as compensatory and bilingual education. Some programs also incorporate 
national data on community characteristics (poverty, income, urbanicity. unemployment, etc.) 
into their allocation formulas. Thus, policymakers have a stake in the validity and reliability 
of the background measures used if they wish to ensure the equitable distribution of Federal 
and State aid payments. 

• Demographic data are needed to gauge the efficacy of particular initiatives. Mans 
education policies and programs are based on implicit or explicit assumptions about the 
background characteristics of students and their communities. Thus, there are major potential 
program implications when such characteristics change over time. l*or example, data on 
potential or actual demographic/student backgiound changes might raise the following kinds ot 
issues for school policymakers: 

• What impact will the increasing prevalence of single-parent and two- 
parent-working families have upon parent involvement in schools? 

• What should be the role of the schools in providing before- and after- 
school care? 

• Given the recent large influx of limited-Fngiish-proficient Asians and 
Hispanies in some areas, how should services be targeted to meet the 
special educational needs of these populations? 

• What are the implications of changes in the health status of American 
students (on such indicators as nutrition, health-related absenteeism. 
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and drug and alcohol use) for education and other social-service 
programs? 

• How should schools, school districts, and States take into account 
changes in rates of student mobility in and out of their jurisdictions 
when designing their programs and services? 

• Demographic data are necessary for determining the "equity" of the education 
system. Whether data are being used for assessment or descriptive purposes, demograpnie 
data are likely to be critical for addressing the broad range of policy issues that fall under the 
rubric of "equity in education." Almost any policy question involving the distribution of 
resources (e.g.. differential tax bases) or instructional practices (e.g.. availability of 
computers or algebra courses) may raise equity concerns if the resources or practices are not 
available to all groups. In order to answer these types of questions, policy analysts must be 
able to examine the distribution of these resources and practices across the various subgroups 
of interests. 

Key Principles and Assumptions 

Before reviewing the status and adequacy of current national collections and the information 
available about student and community background, we will address some technical issues and 
concerns associated with these data. Issues of data validity/reliability, disaggregation, and timeliness 

will b-' discussed separately. 

Validity/Reliability. Any demographic measure should be pretested and evaluated to 
ensure that it measures what it purports to measure. Inconsistency in how a measure is 
defined and interpreted by respondents will seriously compromise its usefulness. For 
example, the item from the National Center for Education Statistics (NCES) High School and 
Beyond survey, "Were you born in this country?," is not valid for Puerto Ricans because of 
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Puerto Rico's Commonwealth status. 1 Similarly, because different States use data for 
different purposes, their natural tendency is to tailor data definitions to their needs. Thus, 
States may define "at-risk," "poor," and even "enrolled" students differently, mitigating 
comparisons across these jurisdictions. The trick is to devise and implement standard 
definitions that provide the vast majority of States with the information they want while also 
allowing for reliable national reporting, fund-allocation decisions, and interstate comparisons. 

In addition, it is important to eliminate any obvious threats to the validity/reliability of 
demographic data resulting from possible incentives for respondent dishonesty. For example, 
guarantees of respondent confidentiality should be made explicit in all collections, and the 
data collection agency should take all necessary steps to ensure that these commitments are 
maintained. 

Disaggregation. It is appropriate to discuss two levels of detail in presenting the student 
and community background data: (I) simple descriptive aggregates at the State or national 
level and (2) variables related directly to school process, resource, and outcome measures at 
the student, school, and school district levels. Aggregate nationally collected, descriptive 
characteristics are useful in describing States and the Nation on broad characteristics of 
students and their home communities assumed to affect the conditions and outcomes of 
schooling, ft is also particularly important to monitor trends in these characteristics over 
time, both to provide a context for other data reported on trends in schooling (e.g., changing 
resource levels or student outcomes) and for planning initiatives based on changing 
demographic conditions. 



1 Fortunately, subsequent analyses and telephone followups reveal that Puerto Rican students 
overwhelmingly did not consider Puerto Rico to be part of the United States when answering this question 
(Fernandez, 1983). 
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Linkages between student and community background variables and school process, 
resource, and outcome measures at lower levels of aggregation (student, school, school 
district) are particularly useful for addressing questions of equity and equal education 
opportunity. Since any policy question (e.g., how well are students achieving academically?) 
has a corresponding equity version of the issue (e.g., how does academic achievement differ 
for students with different family frcome levels, and how is this pattern changing over time?), 
data collected to answer the former question can be used to address the latter question as well. 
In reeognition of this fact, it is important to provide data on as many policy-relevant groups 
as possible in reporting on the conditions and outcomes of schooling. 

Timeliness. For data to be most useful for policymakers, they must be up to date. 
Especially when dealing with rapidly growing or mobile groups, or dynamic economic 
circumstances, "old" data may yield flawed conclusions. Because of this, linking data on 
school resources, processes, and outcomes with dated Census data as well as using such data 
for fund allocation purposes may be problematic because Census information will be outdated 
for broad spans of time. While this does not mean we should ignore available Census data, it 
does imply that careful considerations should be given to the degree of reliance on decennial 
census counts. Other more frequent collections of demographic information on either a 
sample or universe basis appear to be in order. 

Decennial census data can probably best be used when the objective is to analyze, in the 
aggregate, broad community demographic characteristics (such as wealth, income, density, 
and racial/ethnic composition) in relation to the aggregate education resource levels, school 
processes, and outcomes occurring in these communities. The fact that the reported 
community demographic characteristics may be out of date is not critical here because the 
focus is on the relationships between the demographic data and these other factors. As long 
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as the background data are collected for a similar time period as that of the data to which they 
are being related (or the data can be reasonably assumed to represent the background 
conditions of that period), useful analyses can proceed. The NCES-sponsored Census 
Mapping Project, which relates decennial census data to school district boundaries, will permit 
such analyses using 1990 census data (NCES and U.S. Bureau of the Census, 1989). 

When should decennial census collections be considered too infrequent for policy use? If 
the goal is to monitor the prevalence and distribution of critical student-population 
characteristics and how they are changing over time, decennial census data are usually 
inadequate. Given the shifts in population characteristics over a decade, data on student and 
family demographics should certainly be collected mure than once every 10 years. However, 
except when used for allocating funds, such collections probably do not need to be carried out 
every year, either. Experience with the General Social Survey, an omnibus social-indicators 
survey conducted by the National Opinion Research Center (NORC). may be instructive. 
After initially conducting the survey on a yearly basis, the survey's sponsors (the National 
Science Foundation) concluded that fielding the survey every two years provided information 
that was both cost-effective and highly useful, 

B. Current National Data Collections: Their Limitations and 
Potential Strategies for Improvement 

The availability of data on student and community background characteristics varies greatly by 
data source. For example, sample surveys of individuals such as the National Educational 
Longitudinal Survey (NELS) (NCES 1990c-d), the National Assessment of Educational Progress 
(NAEP) (Educational Testing Service 1989), and the Current Population Survey (CPS) (Bureau of the 
Census 1989a-b. and 1990a-b) have collected significant amounts of data on background 
characteristics of individual students. On the other hand, the Common Core of Data (CCD) collects 
28 



aggregate data from the public school universe on a more limited number of background variables 
(NCES 1988 and 1990b). It is appropriate that there are differences in the level of detail of the data 
that are collected. A sample survey imposes a greater burden on a much smaller number of 
respondents, who usually have immediate access to the information being requested. A data 
collection of the scope of the CCD must seek only the aggregate types of information that the 
respondents, who are using administrative data, can readily access. 

The following two sections discuss in greater detail the types of student and eommi-.iity 
background data currently available as State/national-level aggregates (usually through school universe 
collections) and data that can be directly related to other measures of schooling at the student, school, 
and schtHil district levels (usually through sample surveys). Areas where current collections must be 
improved to address policy questions discussed earlier are particularly noted. 

I. National and State Aggregates 

The main source of annual State and national aggregate data about student characteristics is 
NTKS* CCD surveys (NCES 1988 and 1990b). These are annual universe collections from State 
administrative record systems that report descriptive information about the numbers and broad 
characteristics of all public school students and staff. Most reports are at the State-aggregate level, 
although public-use data tapes are available containing data on schools and school districts. 

School enrollment counts have historically been available from the CCD in two ways: as 
Average Daily Attendance (ADA) and as fall membership counts. Aggregate State ADA reports 
generated from the fiscal survey component of the CCD collection are part of a legislated formula 
used to generate Federal allocations for Chapter 1 and other programs. However, the comparability 
of State ADA counts has recently been called into question by studies of the Council of Chief State 
School Officers (CCSSO) (Clements et al. 1988). the Inspector General of the U.S. Department of 
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Education (Office of Inspector General 1989), and NCES itself (NCES 1990a). The Inspector 
General and NCES have recommended changing the relevant Federal allocation formulas to use the 
NCES fall memhership count--a more comparable statistic across States that is collected through the 
nonfiscal component of the CCD surveys-rather than ADA. 

The CCD currently provides only limited information on student participation in 
prekindergarten programs, an increasingly important issue as evidenced by the national readiness goal 
recently endorsed by the Nation's Governors and the President/ Only participation in programs 
operated by public school systems are included in the prekindergarten count. Participants in privately 
funded programs and programs managed by other agencies (e.g.. community service agencies that 
operate programs such as Head Start) are not included. 

Another area of increasing attention among policymakers is the count of students with special 
educational needs. In recent years, the CCD surveys have been expanded to collect information on 
student race/ethnicity, tree-lunch eligibility (a surrogate indicator of socioeconomic status), and 
special education status. However, the absence of free-lunch and racial/ethnic counts in many States, 
as well as the inability to break down the numbers of special education students by type of 
handicapping condition, have thus far limited the usefulness of these data. In addition, although the 
CCD collects overall enrollment levels tor each grade, data on race/ethnicity (as well as sex) cannot 
be broken down by grade level. 

Despite widespread interest in determining the number of pupils whose first language is other 
than English, the CCD does not collect any data on this condition. Nor does the CCD report on the 
number of students participating in bilingual or vocational education programs or who receive 
compensatory education services. 

- The goal states that by the year 2000, all children will start school "ready to learn." One of the jointly 
endorsed objectives listed under this goal is that "...disadvantaged and disabled children have access to 
high quality and deveiopmentally appropriate preschool programs...." 
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Two other areas of aggregate student background data that education policymakers would And 
useful are statistics on health status (e.g., nutrition, health-related absenteeism, and drug and alcohol 
use) and mobility rates across different schools, school districts, and States. The CCD does not 
currently provide data in these areas. 

In many instances, other Federal agencies may either collect or plan to collect data that would 
address some of the shortcomings that we have noted in the CCD. The Office of Civil Rights (OCR) 
collects national (but not State-level) data on racial/ethnic affiliation. The Office of Special Education 
and Rehabilitation Services (OSERS) collects data from each State on student handi- apping conditions, 
although the data are not reported by race and specific handicapping condition, which would be 
particularly valuable. Counts of compensatory, bilingual, and vocational education students are 
collected by various program offices within the U.S. Department of Education. The Food and 
Nutrition Service (FNS) of the U.S. Department of Agriculture collects State-aggregate data on free- 
lunch participants. The Administration for Children, Youth, and Families (ACYF) within the 
Department of Health and Human Services collects data on Head Start participants, and the Centers 
for Disease Control (CDC) is launching a study to obtain national estimates of student participation in 
"at-risk" behaviors. 

Finally, the Census Bureau's Current Population Survey (CPS), a traditional source of 
annually reported national and regional demographic data, may be expanded soon to provide State- 
representative data. Among other things, this data base could potentially provide annual estimates of 
within-State and interstate student mobility patterns. The U.S. Department of Education's annual 
collection on the number and location of migrant students can provide useful, though more limited, 
data on this general question. 

Wherever data from other Federal sources could be used to buttress NCES aggregate student 
and community background data, it is important to ensure adequate coordination among the relevant 
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agencies in the collection and reporting of these data. Improving coordination in data collection and 
reporting will result in a more comprehensive picture of aggregate student demographic conditions 
and trends. It can also provide data users with a more systematic identification of areas where 
information "gaps" remain and reduce the collection burden on data providers. 

One should be aware, however, that improved Federal-level coordination of student and 
community background collections will sometimes be a difficult objective to achieve. Agencies 
collect and report data in certain ways to fulfill their own perceived statutory mandates and are often 
hesitant to alter their policies when they believe that their primary mission could, as a result, be 
compromised. The Food and Nutrition Service, for example, is currently opposed to releasing free- 
lunch information below the school district level because it feels that such releases would jeopardize its 
client's trust in the confidentiality of the information they provide. This policy effectively limits the 
broader use of this measure by education policymakers and researchers who consider it extremely 
valuable as the only readily available indicator of student socioeconomic status (SES). 3 Achieving 
better data coordination thus may often require extended negotiations and. ultimately, compromises 
among all affected parties so that divergent agency objectives can be accommodated. For this reason, 
and also because the validity and reliability of free-lunch data as an SES surrogate has sometimes 
been questioned, it is critical that the Federal government thoroughly investigate issues associated 
with the use of the free-lunch statistic by the education community as well as potential alternative 
measures of student socioeconomic status. 

Another potential source of data on student and community background characteristics is the 
NCES Schools and Staffing Survey (SASS) (NCES 1989). First administered in 1988, SASS contains 
State-representative sample data on school enrollment (by grade), poverty levels (based on free-lunch 
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' The policy should not. however, interfere with the regular collection and reporting of aggregate State- 
level free-lunch data. 
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counts), and racial/ethnic composition. 4 These can be used to give policymakers State-level estimates 
of broad demographic trends over time on these dimensions. (There are plans to administer this 

survey every two years after 1991.) 

Data from SASS could also be used to provide similar background information on students 
enrolled in pri.^e schools. Unfortunately. SASS private school data (unlike public school data) 
cannot provide representative State estimates, and the validity and reliability of private school 
estimates of student background characteristics may be problematic/ The NCES Private School 
Survey-conducted every two years-collects fall enrollment data by grade level from the known 
universe of private schools. No information is currently available from this survey on the background 
characteristics of private school students. 

2. Variables for Direct Linkage to Education Characteristics 

NCES' two premier student-level data collection eflorts-NAEP and NEl.S:88-provide data 
on many student background variables including type of school attended (public/private), 
race/ethnicity, sex. age. language spoken, attitudes, and socioeconomic status (Johnson and Zwick 
1990. NCES 1990c-d). The current NELS instrument provides more data of this type than does 
NAEP and is probably more reliable because it contains information from parents as well as students. 
NELS is certain to become a rich source of analytic information relating student instructional 
processes and outcomes to demographic characteristics, in cross section and over time. NAEP also 
reports at least some of its achievement data every two years and is planning to expand its assessment 
to include State-representative data (for those States choosing to participate) in the coming decade. 



4 Race/ethnic estimates can also be made at the school district level from the SASS. 

> Free-lunch counts, for example, are probably less valid indicators of school poverty in private schools 
since many private schools do not participate in the Federal school lunch program even though they serve 
students who meet the eligibility criteria for participation. 



Also under discussion for the 1992 NAEP is some expansion in the number of student background 
questions to be included in the survey. For example, the inclusion of a variable on student 
mobility/school transfers is being discussed. 

In addition to NELS:88 and NAEP, SASS data on student and community background 
characteristics can be particularly useful for school-level analyses. It is possible, with SASS. to relate 
the demographic composition of schools as reported by teachers and school administrators to 
important instructional features, particularly curriculum offerings and teacher competencies. 

A new fiscal component of the CCD, to be collected in 1990 through a joint NCES/Census 
effort, will provide information about school district revenues and expenditures. Such data can be 
linked to decennial census information on the demographic characteristics of communities to 
investigate questions of resource adequacy and equity, 

C. Summary of Recommendations for Improving the Collection and 
Reporting of Student and Community Background Statistics 

The policy questions that can be informed by demographic data h*ve a number of important 
implications for data collection and reporting. Most fundamentally, policymakers need the ability to 
ascertain broad trends and patterns over time on key demographic characteristics of students, families, 
and school communities. Given the mobility of student populations and the frequent changes in their 
circumstances, data on such characteristics need to be collected often and reported with regularity. 

In addition, there is the issue of allocating resources '•fairly" based on the relevant input 
characteristics of students. When jurisdictions employ idiosyncratic definitions of student 
characteristics (e.g., race, income, attendance) that are used to allocate education program funds, the 
integrity and fairness of the programs and their funding systems are compromised. Thus, whenever 
demographic data are used to allocate program funds, definitions should be consistent and uniformly 
applied. 
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Finally, since demographic data are likely to be related to other information in performing 
many analyses, policymakers need to be able to look at variables of interest by demographic 
subgroup. Whether a policy question is put in terms of individuals (e.g., are students receiving 
instruction from "qualified" teachers?) or aggregates (e.g.. are schools and districts employing 
appropriately "qualified" instructors?), the equity question of whether the findings are consistent for 
all racial/ethnic groups and social classes is relevant. Thus, regardless of the manner in which a 
question is posed, student and community background data are critical for addressing questions of 
equity. 

The following recommendations in the domain of student and community background statistics 
represent broad directions for statistical improvements that are designed to better meet the data needs 
of the education policy community. It is assumed that any statistical products ultimately created in 
response to these recommendations would have to pass the tests of data validity, reliability, level of 
aggregation, and timeliness discussed in Chapter 1 if they are to be truly useful to policymakers. To 
achieve these objectives, continued cooperation among data providers and users will be necessary in 
the developmental work that lies ahead. 

The National Forum makes the following seven recommendations tor improving data 
collection and reporting in the domain of student and community background statistics: 

1 . Using data extracted from State administrative record systems on the universe of 
public school students, the National Center for Education Statistics (NCES) should 
annually collect and report State- and national-level aggregates on the following 
student background characteristics: 

• Fall membership counts by race/ethnicity by grade; and 

• Fall membership counts by sex by grade. 
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NCES should annually report State- and national -aggregate statistics collected by other 
agencies on the following student subgroups: 

• Handicapped students served, by type of handicap; 

• Free-lunch participants; and 

• Participants in compensatory, bilingual, and vocational education programs. 

NCES, in cooperation with other Federal and State agencies, should work toward the 
regular collection and reporting of the following State and national student background 
statistics: 

• Limited-English-proficiency status; 

• Student handicapping conditions by race; 

• Participation in prekindergarten education programs; 

• Student health status (e.g., nutrition, health-related absenteeism, and drug and alcohol 
use); and 

• Student mobility and migrant status. 

The Office of Educational Research and Improvement (OERI) should fund special studies 
investigating the efficacy of using free-lunch data as proxies for student socioeconomic 
status (SES), and the costs, benefits, and burdens associated with regularly collecting and 
reporting alternative SES measures. These studies should specifically examine issues of 
validity, reliability, and usefulness of free-lunch and alternative measures for different 
types of reporting and analysis as well as administrative issues related to the collection and 
reporting of such measures. 

NCES should develop the capacity to collect and report data on private school student 
background characteristics that are parallel to those being developed for the universe of 



public school students. Data might come from the NCES Private School Survey and the 
Schools and Staffing Survey, and they should be reported as national aggregates and, to 
the extent feasible. State aggregates 

In reporting measures of education resourc e, school processes, and student outcomes from 
its sample and universe surveys. NCES should attempt, to the extent feasible and 
appropriate, to provide disaggregated data using the following student and community 
background characteristics: 

• Sex; 

• Racial/ethnic-group affiliation; 

• Limited-English-proficiency status; 

• Community wealth; and 

• Family income. 

NCES should consider reporting distributional patterns for the following student and 
community background variables in conjunction with particular resource, process, and 
outcome measures: 

• Public/private school enrollment; 

• Student employment status; 

• Measures of family background (e.g., parents' education, language spoken in the home); 

• Student mobility; and 

• Student handicapping condition. 
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Chapter 3 
Education Resource Statistics 



A. Rationale and policy question* - p. 41 

B. Current data collections and their limitations - p. 43 

C. Recommendations for improving collection and reporting - p. 55 



This chapter is based on an "idea paper" prepared for the National Forum by Dr. Margaret 
Goertz of the Educational Testing Service (ETS), Princeton, New Jersey. Dr. Goert/,, assisted 
by Dr. Martin Orland of the National Center for Education Statistics, also prepared the draft 
manuscript for the chapter in eiwiperation with the Forum subcommittee on education resource 
statistics chaired by Dr. James Phelps of the Michigan Department of Education. 
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Education Resource Statistics 



Education resources include both money, fiscal resources, and those resources that money 
buys, human and nonhuman resources. States-and school districts within States-have varying 
amounts of money or revenues available to them, governmental levels providing funds (e.g., Federal. 
State, intermediate, and local), and funding sources (e.g., taxation, aid, nontax revenues). For 
example, average State per-pupil expenditures ranged from a high of $8,253 to a low of $2,362 in 
1985-86 (NCES 1988, Table 1 15). Sources of funding vary as well. The State of Hawaii provides 
nearly all elementary and secondary education resources, while New Hampshire provides less than 10 
percent (NCES 1988, Table 108). 

States, local education agencies (LEAs)* and schools use fiscal resources to obtain the human 
and nonhuman resources required to provide education sen-ices. Human resources include 
instructional and noninstructional staff; nonhuman resources include facilities, textbooks, computers, 
and computet software. The amount, type, and mix of human and nonhuman resources that a district 
purchases is influenced by district objectives. State mandates, and cost differentials. 

A. Rationale and Important Policy Questions 

Education policymakers and the public have become increasingly interested in recent year* in 

how education dollars are spent and in the relationships between education revenues, expenditures, 

and outcom They seek information that will address the following questions: 

1. What is the total amount spent on elementary and secondary education at the 
national, State, and local levels? 

• How has this level of support changed over time? 
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2. What percentage of that amount comes from each source of revenue (i.e., Federal, 
State, intermediate, local, private)? 

• How has the level of education revenues changed in relationship to the national 
economy (as measured by personal income. GNP/Gross State Product {GSP])? 

• How has the relative contribution of each source changed over time? 

• How have the sources of local revenue changed over time (e.g.. changing reliance on 
the local property tax to support education)? 

3. What do education dollars buy at the national, State, and local levels? 

• How much is spent on instructional services? Noninstructional services? 

• How much is spei:? on human resources? Nonhuman resources? 

• To what extent have new dollars purchased additional services? Supported existing 
services {e.g., increased teacher salaries)? 

• How have these patterns changed over time? 

4. How are education resources distributed union)* the States and school districts? 

• To what extent does the level of education resources vary among States? LEAs? 
How have these differences changed over time? 

• How does the mix of education revenues vary across States and LEAs? 

• How does the mix of expenditures by function differ among States? LEAs? 

5. How do States allocate education resources given differences in levels of student need, 
fiscal capacity, and costs? In particular: 

• How much are States spending for teachers and other education staff, taking into 
account these differences? 

• How much revenue is raised from Federal. State, and local sources, taking into 
account these differences? 
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B. Current National Data Collections: Their Limitations and 
Potential Strategies for Improvement 

The Federal Government collects and reports some national- and State-level education 
resource data on an annual basts. The National Center for education Statistics' (NCES) Common 
Core of Data (CCD) surveys and the Census Bureau's Survey of Local Government Finances are the 
primary Federal sources of information on fiscal and nonfiscal education resources. Because the 
CCD data are used more extensively by education policymakers, this section will focus on the CCD 
education resource collections. 

CCD Surveys anc| Publications 

Fiscal Survey. The CCD fiscal survey, completed every year by State CCD coordinators, 
collects information on education revenues and expenditures, aggregated across all public elementary 
and secondary school districts in each State. Data from this survey are published in Federal reports 
and used in calculations to determine the funding allocations for a number of Federal education 
programs. 

In recent years, the CCD fiscal survey requested revenue data aggregated into only four 
categories (Federal, State, intermediate, and local) and expenditure data aggregated into only three 
functions (instruction, support services, and noninstruetional services). However, with the 
implementation of the new "National Public Education Financial Survey" in FY 1989. the CCD fiscal 
survey, for the first time in almost a decade, requested detailed fiscal information on revenues and 
expenditures. 

For example, States were asked to report on 13 different sources of local revenues and to 
report on instructional and support service expenditures by 6 object categories. The number of 
functional areas has been expanded as well. In addition, the new CCD fiscal survey uses the same 
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classifications that appear in the NCES 1980 Financial Accounting for Local and State School Systems 
(Handbook 2R2). 

Although NCES has collected aggregated education fiscal data from States for many years, the 
data in each of the major reporting categories have not necessarily been comparable across States. 
The degree of comparability is determined by the similarity of fiscal definitions and/or procedures 
States use when reporting data. A survey conducted by the Council of Chief State School Officers* 
Education Data Improvement Project (EDIP) revealed that only 23 States use accounting systems that 
conform closely to Handbook 2R2/ Twenty-five States use earlier versions of the handbook (17 use 
a handbook similar to Handbook 2R |1973], and eight use a handbook similar to Handbook 2 |1957|). 
The remaining three use their own systems (Education Data Improvement Project 1988). 

One of the primary objectives of the Education Data Improvement Project was to compare the 
definitions and classifications used by each State with those contained in Handbook 2R2. The project 
staff found that States appear to collect data on State and Federal revenues that conform closely to 
Handbook 2R2 definitions and at approximately the same level of detail. However, States are less 
consistent in collecting local revenue data. In particular, many States do not break out information 
about local tax, tuition, and transportation sources by subcategories or do not collect data from LEAs 
using these subcategories. 

With regard to education expenditures, most States can provide comparable data for the 
functions under instruction, support services, noninstructional services, and debt service. However, 
State data on capital outlays by object categories and on expenditures made by Slates on behalf of 
LEAs (e.g., textbooks, transportation, pensions) are more problematic. Fewer than half of the States 
collect expenditure information about functions under capital outlays, and many States do not 



A The project assigned States to a particular handbook if at least 80 percent of the definitions in the State 
accounting handbook agreed with the definitions in the Federal handbook. 
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distinguish between instructional and noninstructional staff categories when reporting salary and/or 
employee benefit data. There is some evidence, however, that more detailed expenditure information 
is often available at the object level from LEAs (Education Data Improvement Project 1988). 

NCES has initiated efforts to help States address the comparability problem. The Center 
provides training to SEA personnel who complete the CCD fiscal survey during its annual Elementary 
and Secondary Education Data Conference each summer. In addition, the NCES "Crosswalk Project" 
has made it possible to "crosswalk," or recombine, noncomparable fiscal items into the correct 
reporting categories as specified in the new fiscal survey. The project identified areas where State 
definitions and/or procedures differed from those specified for the new fiscal survey, enabling NCES 
to tailor training for State data coordinators and also to develop crosswalk protocols for State 
respondents to use when completing the CCD survey (NCES 1989). 

The expanded CCD fiscal survey (with support from the "Crosswalk Project" and data from 
the CCD nonfiscal surveys) will enable NCES to provide most of the data needed to answer the 
education resource questions posed above at the National and State leveis-but not at the local level. 
For example, to address the question of how education resources are distributed in the different 
States, the revised survey will permit the reporting of State expenditure patterns by "function" 
(e.g., amounts spent on instruction, general administration, school administration, etc.) by "object" 
(e.g., amounts spent on salaries and employee benefits). The survey will also make it possible to 
report objects within a function or functions within an object (e.g., percentage of salary costs going to 
instruction or, conversely, the percenlige of instructional costs spent on salaries). A more complete 
listing of the data elements in the revised CCD fiscal survey is provided in an appendix to this 
chapter. 

Nonfiscal Survey, As noted in Chapter 2, the CCD nonfiscal surveys collect data at the 
school, district, and State levels on the number of students and staff in the public elementary and 
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secondary school system. Student data collected at the school level include membership counts by 
grade, racial/ethnic grouping, and free-lunch eligibility status. 7 The membership definition used in 
the 1988-89 survey is the count of students made on the school day closest to October 1. Counts of 
students with special education individual education plans (IEP) are currently available at the school 
district but not school level. However, the number of students receiving special compensatory or 
bilingual education services are currently unavailable on any level from these surveys. 

Staff data collected in the nonfiscal survey include numbers of teachers (prekindergarten, 
kindergarten, elementary, secondary, and ungraded), instructional aides, counselors, librarians, school 
and district administrators, and support staff. Counts of these are available at the State-aggregate 
level only, except for the number of classroom teachers, for which school-level data are collected. 

Education Resource Publications- The U.S. Department of Education has reported trends in 
the level and types of education revenues and expenditures for many years. For example, the Digest 
of Education Statistics (NCES 1988) reports the following information for the current year and for 
selected prior years: 

• Revenues for public elementary and secondary schools by source of funds for the Nation 
and by State; 

• Current expenditures for education by purpose for the Nation; 

• Current expenditures for education by State; 

• Current expenditure per student for the Nation and by State; 

• Average teacher salary for the Nation and by State; 

• Index of public school revenues in relation to personal income; and 

• Number of staff and pupil-teacher ratios. 



7 As discussed in Chapter 2, data are currently not available from some States on racial/ethnic counts 
and free-lunch eligibility by school. 

46 

so 



In addition, resource data collected by NCES are reported in E.D. TABS, topical bulletins 
and analysis reports, and the Education Department's State Education Performance Chart (the "Wall 
Chart"). These reports enable State and Federal policy analysts and school finance researchers to 
track changes in the financing of public elementary and secondary education and to understand how 
different States finance education* 

As data from the expanded CCD collection are received, NCES will report more detailed 
information on the components of education revenues and expenditures at the national and State 
levels. These new data will permit NCES to report alternative measures of elementary and secondary 
education expenditures such as "core* education expenditures; data on selected expenditure functions 
such as instruction, administration, operations and maintenance, and transportation; and the 
relationships among State contextual factors and fiscal measures (e.g., the relationship between per- 
capita income, gross State product, and percentage of poor students and/or core expenditures per 
pupil) (Moore, Myers, and Gutmann 1989), 

Limitations 

We have identified 10 specific limitations in the current NCES collection and reporting of 
education resource data. 

1. Cost Adjustments. We believe that it is critical to include some type of cost adjustment 
factor when comparing fiscal resources across communities and States. Similar expenditures will 
purchase different units of the same resource in different communities and/or in different States. For 
example, the average teacher salary in Alaska is $44,000 compared with $19,500 in Mississippi 
(NCES 1988, Table 58). Thus, the same number of dollars can support twice as many staff members 
in Mississippi as they can in Alaska. 
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It is necessary to account for differences among States in the cost of providirg similar 
education services in order to avoid potentially misleading impressions created by national resource 
statistics. For example, the expenditure and salary statistics presented in the Education Department's 
"Wall Chart," while appropriate for inclusion as resource measures because of their perceived value 
and importance, are not truly comparable across States because no cost adjustments are made. Nor 
can trends over time in education resource commitments be determined (either for individual States or 
nationally) as long as the measures used are not convened from current to constant dollars. ITie need 
for State specific education resource cost adjustments should become even more critical in the future 
with the release of State-by-State NAEP results because the press, public, and policy analysts may 
relate State expenditure figures to tested achievement results. 

If cost adjustment techniques were developed, national statistical publications such as the Wall 
Chart could use them to enhance the comparability of the following currently reported measures: 
(1) average teacher salary; (2) current expenditures per pupil; and (3) expenditures for classroom 
teachers as a percentage of total current expenditures. Cost-adjusted expenditure and salary data 
could also be incorporated into regular NCES publications when appropriate." 

2. International Cost Comparisons. The recent publicity associated with the publication of 
a report comparing education expenditures levels in the United States with those of other 
industrialized nations (Rusell and Mishel 1990) and the reaction to that report by the U.S. Department 
of Education exemplify the growing interest in international education resource comparisons. Many 
of the difficulties associated with developing a truly comparable international resource measure are 
similar conceptually to those that arise in making valid interstate resource comparisons (such as 
controlling for differences in the unit costs of "equivalent" teachers or other resource inputs). Others 



8 We would recommend that these newly "adjusted" measures be reported as additions to, not 
replacements for, the current unadjusted statistics in these areas. 
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are more unique to the international arena (e.g., taking into account the vastly different roles of 
private-sector education service providers and distinctions between elementary-secondary and 
postsecondary education). Clearly, the problems of both conceptualization and implementation are 
substantial. However, given the level of interest in such a statistic (one, we suspect, that is likely to 
continue as international comparisons in other education domains become more commonplace), there 
is a need to begin the necessary research and development work to determine whether a comparable, 
regularly reported international statistic on education resource levels can be developed. 

3. Student Counts. Because pupil counts are essential for presenting education resource 
levels on a "per-unit" basis (the unit being the pupil), counts of students are critical elements of 
education resource data. As noted in Chapter 2, NCES collects two types of student counts: fall 
membership (Nonfiscal Surveys) and Average Daily Attendance (ADA) (Fiscal Survey). Federal law 
requires that ADA be used to calculate the State-per-pupil expenditure (SPPE) for certain Federal 
program allocations. 9 States may use one of two methods in reporting ADA: (1) ADA as defined by 
State law or (2) only in the absence of a State definition, ADA as defined by NCES. 

The use of ADA as a way to measure student counts can create several comparability 
problems. First, the denominator in the per-pupil expenditure figure reported for each State is not 
comparable because the definition of ADA varies among States. Second, the use of ADA in reporting 
expenditure and revenue data limits comparability with other education resource data such as teacher- 
student ratios, where membership is the denominator. 

Another data limitation concerns counts of special-needs students. As reported in Chapter 2, 
NCES does not collect or iegularly report data on students who participate in special-needs programs 



' These programs include Chapter 1, Impact Aid, Indian Education, Part B of Education of the 
Handicapped, Title VII of the McKinney Homeless Assistance Act, and other programs whose allocations 
are based, in whole or in part, on the State per-pupil expenditure data derived from the information 
reported by SEAs (NCES 1989 p. 27). 

49 

no 



such as ECIA Chapter I and other compensatory education programs, special education, bilingual 
education, and vocational education. Yet these data wi!l be reeded to address policy questions 
relating resource allocation levels to differential levels of student need (See policy question No, S in 
this chapter). 10 

4. Program Expenditures. Current national resource data collections provide no 
information on relative financial commitments for particular program purposes, or for different 
curriculum content areas and grade spans. It wi.uld be valuable to know how much States and the 
Nation spend for such purposes as compensatory education, special education, and vocational 
education, as well as how these expenditures are changing over time. It is also important to learn 
about and monitor resource commitments in such areas as science, math, and foreign-language 
education, particularly at the secondary level. 

We recognize that developing a complete capacity to collect and report data of this type is a 
long-term proposition, requiring the creation of program-based accounting systems and comparable 
definitions of relevant program categories in different locales. Nonetheless, it might be possible to 
estimate major costs for particular programs without a full-scale transition to program accounting. 
For example, secondary school teacher salary expenditure, oy subject area might already be available 
in many States or require only relatively modest refinements in existing State recordkeeping systems. 

5. Teacher Salaries, Until quite recently, no information on elementary/secondary teacher 
salary levels in different States has been available from government sources. Since teachers constitute 
the primary resource input in education, it is important for policymakers, analysts, and researchers to 
know how the costs of this input vary among States, how the costs may be hanging over time, and 
how the costs relate to other features of schooling within the broad education context. An average 
teacher salary measure would reveal the general level of resources committed to teaching personnel, 

10 Recommendations 2 and 3 in Chapter 2 of the Guide attempt to address this current data limitation. 
50 
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and a measure of teacher salaries with specified degree and experience credentials would reveal the 
level of resources committed for comparably classified personnel. Over the long term, measures 
indicating the total dollar investment in education personnel-broken down by type of staff and 
expenditure (e.g.. salaries, fringe benefits)--would serve a wide range of purposes. 

The Schools and Staffing Survey (SASS) provides representative State-level estimates of 
average teacher salary levels as well as of major salary components (base pay. summer pay. 
extracurricular pay, etc.). Such data, while vital as an analytic variable in the SASS data set. are 
more problematic as regularly rer <led "stand alone" indicators of teacher salary levels in a State. 
Because of widespread interest in this statistic, it has typically been reported annually by the National 
Education Association and the American Federation of Teachers (the only current sources of State- 
lev uata on teacher salaries). SASS data, however, are reported only every other year (after 1991). 
Further, because they are from a representative sample of teachers, the SASS data mtain sampling 
errors that might make it difficult to monitor and compare State-level trends over short periods or to 
relate SASS findings directly to other reported items on the CCD surveys (such as overall State per- 
pupil expenditure levels). Given these concerns, expanding the CCD surveys to incorporate teacher 
salary data would seem to be a preferable strategy. 

6. Data on State Revenues. Tht expanded CCD finance survey does no collect detailed 
information on State aid allocated to local school districts such as general aid. categorical aid. and 
building aid. Yet this type of information would provide policymakers and analysts with a better 
understanding of the ways that States finance education in general and education reform efforts in 
particular. These data could be obtained by including the following categories of State aid in the 
NCBS annual fiscal survey: 



• General aid; 

• Categorical aid (e.g., compensatory, special, bilingual, vocational education); 

• Pupil support services (e.g., transportation, textbooks); 

• Building aid; and 

• Direct program support on behalf of LEAs (e.g.. textbooks, transportation, employee 
benefits). 

7. Private School Expenditures. Financial data on private school expenditures is currently 
absent from regular national resource collections and reports. NCES periodically surveys private 
schools to collect data on the number of schools, students, and staff. The primary source of 
expenditure data is the National Catholic Educational Association, which regularly publishes reports 
on Catholic schools and their finances (National Catholic Educational Association 1988a-b). These 
data are important for understanding the total financial commitment to elementary/secondary 
education in the Nation and individual States. In addition, data on private school expenditures by 
broad categories that are consistent with those collected and reported for public schools would help 
policymakers and analysts understand relative resource commitments in the public and private 
education sectors, how they might change :ver time, and how the types of expenditures may differ 
between the sectors. 

8. LEA Data Collection and Reporting. As noted earlier. NCES currently collects and 
reports fiscal data at the State level only. However, most of the fiscal data reported by the States h;is 
been aggregated from reports submitted by LEAs to their respective State departments of eduction. 
Some States report LEA-level data to the public in district-by-district reports, but others report only 
data aggregated across their LEAs. 

The U.S. Bureau nf the Census collects LEA-level fiscal information annually from State 
education agencies (SEAs) through its Survey of Government Finances (1987). In an effort to reduce 



52 



respondent burden, the Census Bureau uses a variety of data collection procedures including 
reformatting of SEA computer tapes by Census staff, reformatting of computer tapes or computer 
printouts by SEA staff, and compilation from SEA source documents by either Census Bureau or SEA 
staff. 

Every five years (for years ending in two and seven), the Census Bureau report. Finances of 
Public School Systems, provides data on school systems' revenues, expenditures, debt, and financial 
assets for the Nation, for States, and for individual school systems having 5.000 or more 
enrollment." However, the report provdes limited detail on revenue sources and expenditure 
functions,'* and data comparability problems have been identified (Bureau of the Census 1987. p.ix 
and appendix B). 

Collecting Lnd reporting mure detailed and comparable resource data at the LEA level would 
enable education policymakers and analysts to compare expenditures and ievenues among school 
districts within States and/or among groups of districts with different characteristics (e.g., size, 
urbanicity, characteristics of student populations). The Census Bureau and NCES are about to 
undertake a joint local finance collection encompassing all local school districts. It is anticipated that 
the fiscal data obtained from this school district universe collection can eventually be linked to 1990 
decennial census information on district uemographic characteristics. This effort is intended to serve 



" Data tapes containing financial data for all school systems processed in this survey (i.e.. the school 
district universe) are also available from the Census Bureau. In addition, the Census Bureau can supply 
data tapes from its less comprehensive annual load finance collection ( Gaining a mixture of sample and 
school universe data, depending on the State). 

l * The major categories for which data are regularly collected and reported by the Census Bureau are 
intergovernmental revenue, general revenues from own sources (taxes, contribution from parent 
government, current charges, interest earning, ind miscellaneous), intergovernmental expenditures, 
current operation expenditures (instructional services, support services, school lunch, and other), capital 
outlay expenditures, interest on debt, salaries and wages, and debt outstanding. 
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as a prologue to future joint local collections that, at least periodically, would include more detailed 
reporting categories than are currently being requested. 

9. Human Resource Descriptors. The nontlscal CCD collection provides information on 
the numbers of school staff in several human resource categories (including teachers, instructional 
aides, counselors, librarians, and administrators). However, the absence of additional breakdowns 
limits the potential usefulness of the collection. For example, because the "officials and 
administrators" category does not distinguish between personnel providing instructional support 
services directly to students and those whose responsibilities are purely administrative, it is difficult to 
determine the actual distribution of human resource commitments (instructional v. administrative) in 
the schools and how they change over time. 

In addition, the current CCD collection does not contain important descriptive data about 
school staff such as their race/ethnicity, sex, and age. Data of this type are especially important for 
addressing human resource equity issues (e.g.. minority representation in various staffing categories). 
Subject to the limitations cited earlier in the discussion of teacher salaries. SASS can be used to 
provide extensive (State representative) information on the descriptive characteristics of teachers and 
school principals. 

10. School Facility Data. The age and condition of the Nation's school buildings is a 
growing concern among policymakers. Recent national reports have warned that an infrastructure 
crisis may be looming (National Governors' Association 198*)). Yet the State data relied upon in 
making these assessments are incomplete, and their reliability and comparability across locales is 
problematic. Planning at the Federal and State levels would benefit greatly from standardized, 
periodic inventories of the age. condition, and facility needs of school buildings for each State and the 
Nation. 
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C. Summary of Recommendations for Improving the Collection and 
Reporting of Education Resource Statistics 

As discussed above, the Federal Government already collects most of the data needed to 
address the major education resource policy questions. Activities related to the redesign of the 
Common Core of Data already include the development of an expanded data collection instrument that 
provides more detailed revenue and expenditure data and a process of "cross walking" State accounting 
categories into those used in Handbook 2R2. Thus, some of the recommendations concern logical 
improvements or enhancements to current data collection activities. This is not to say that the 
recommendations to further improve the usefulness of the Common Core of Data's fiscal survey will 
be easily or quickly implemented. However, at least in this area, work has begun, and the basic 

direction appears sound. 

In other areas, much developmental work and assessment of alternative strategies will be 
necessary before implementation can proceed. A variety of techniques to adjust resource costs, for 
example, have been developed by economists. Each has strengths and weaknesses, each is 
appropriate for some purposes more than others, and each carries with it different cost and burden 
implications. We should emphasize again that our endorsement of a particular data improvement 
recommendation represents a statement about the potential policy value of a particular kind of statistic 
that is not currently collected or reported. It means that we. as members of the Forum, want to 
jointly investigate this potential data improvement more fully because we believe it cculd fulfill an 
important, and currently unmet, data need. It is not. however, a commitment to support any 
particular plan or strategy for meeting the objectives embodied in the recommendation. 

The National Forum makes the following 12 recommendations for improving data collection 
and reporting in the domain of national education resource statistics. 



The National Center for Education Statistics (NCES) should collect and report a set of 
national- and State-level education revenue, expenditure, and human resource 
measures on an annual basis, using data items from the "National Public Education 
Financial Survey" and the Common Core of Data (CCD) Nonfiscal Surveys. 

NCES should continue to provide training and technical support to States to "crosswalk" data 
elements specified by the current CCD Financial Survey as well 3s other assistance necessary 
for meeting the Handbook 2R2 classifications. 

NCES and other Federal agencies should investigate the feasibility of developing a State-by- 
State statistical measure to adjust education resource data for differences among States and to 
report education resource trends over time in constant dollars. 

NCES and other Federal agencies should investigate the feasibility of developing a State-by- 
State statistical measure to adjust salary data for differences among States and to report 
education salary trends over time in constant dollars. 

NCES and t;ther Federal agencies should engage in research and development efforts that will 
enable them to make accurate, comparable, and informative international comparisons of U.S. 
national education resource commitments with those of other industrialized nations. 

NCES should continue to collect and report data from the CCD aggregated to the State level 
on an annual basis. However, the Center should, over time, develop policies and procedures 
for the regular collection and reporting of district-level resource data. In moving toward 
district-level resource collections, NCES should be particularly cognizant of (1) identifying 
potential reports such data could generate and (2) the capacity of States to provide district- 
level data. 



7. NCES should expand the annual CCD "State Administrative Records Survey" to include: (i) 
an average teacher salary measure that takes into account contract, career ladder, and other 
special incentive pay and (2) a teacher salary measure that takes into account degree status and 
experience. 

8. NCES should make a long-term commitment to establishing a program- and functionally-based 
accounting system. This will provide NCES, policy analysts, and other education researchers 
with better information about how education funds are spent and make it possible to relate 
program resources to the specific education needs of students. The particular program levels 
to be collected should be determined after additional study, taking into account the costs and 
burdens associated with the development of comparable definitions of relevant program 
categories across different locales. 

9. NCES should expand the Federal Government's survey of private schools to include resource 
information. Wherever feasible, the Center should report privaie-school resource data from 
its surveys on a State-by-State basis. 

10. NCES should establish, as a long-term objective, the collection of data regarding the 
status of buildings, including the number, age. condition, and facility needs of the 
Nation's schools. 

1 1 . NCES should regularly report data on the number and descriptive characteristics (i.e.. age. 
sex. race) of instructional, instructional support, and noninstructional staff in the Nation's 
schools. Such data should be reported at the State level to the extent feasible. 
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NCES should establish, as a long-term objective, measures that indicate total dollar 
investments in education personnel. These measures should be specific to different types of 
staff (e.g.. teachers, administrators, instructional aides) and include both direct compensattoi 
expenditures (salaries) and indirect compensation (fringe benefits). 




Chapter 3 - Appendix 



Current Data Elements Provided by the Redesigned 
Common Core of Data Financial Survey 

Except where indicated, the data elements listed below will come from the expanded Common 
Core of Data (CCD) fiscal survey. They are grouped according to the policy questions noted at the 
beginning of this chapter. 

1. What is the total amount spent on elementary and secondary education at 
the (national and Statel levels? 

2. What do education dollars buy at the [national and State! levels? 

3. How are education resources distributed among the IStatesJ? 

• Education expenditures 

■ Total 

■ By object 

• Salaries 

• Employee benefits 

• Purchased services 

• Property and supplies 

• Other 

• Expenditures on instruction 

■ Total 

■ By object 

• Salaries 

• Employee benefits'* 

• AliOther' 4 
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1? Includes direct State funding of employee benefits tor public school employes. 
14 Includes direct State funding of textbooks for public school children. 
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• Expenditures on support services 

■ Total 

■ By function 

• Students 

• Instructional staff 

• General administration 

• School administration 

• Operations and maintenance 

• Student transportation 25 

• Other 

• NoninstructionaJ expenditures 

■ Total 

■ By function 

• Food services 

• Enterprise operations 

• Student body activities 

• Community services 

• Capita] outlay and debt service 

■ Total 

■ By function 

• Facilities acquisition and construction 

• Equipment 

• Debt service: principal and interest 

• Direct-cost programs (expenditures for other than public pre-K-I2 programs) 

■ Total 

■ By function 

• Nonpublic school programs 

• Adult education programs 

• Community college programs 

• Other 



55 Includes direct State support of transportation for public school students. 
60 
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• Number and types of staff providing education services 

■ Teachers 

■ By Level 

• Prekindergarten 

• Kindergarten 

• Elementary 

• Secondary 

• Ungraded 

■ Other Staff 

• Instructional aides 

• Counselors 

• Librarians 

• School administrators 

• District administrators 

What percentage of {the amount spent on elementary and secondary 
education] comes from each source of revenue (e.g., Federal, State, 
intermediate, local, private)? 

• Total education revenues 

• Total Federal education revenues 

• Total State education revenues 

• Total intermediate education revenues 

• Local education revenues 

■ Total 

■ By category of revenue 

• Local property taxes 

• Nonproperty taxes 

• Revenues from other local governmental units 

• Tuition and transportation from individuals and other LEAs 

• Food service 

• Student activities 

• Other local revenues 
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5. How do States allocate resources given differences in levels of student 
need, fiscal capacity, and costs? 

a Fiscal capacity 

■ Personal income"' 

■ Gross state product per school-aged child" 

■ Relative tax capacity'" 

• Student counts 

■ District enrollment (nonfiscal surveys) 

■ Average daily attendance (fiscal survey) 
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From ihe U.S. Census Bureau's annual Current Population Survey. 

From the U.S. Bureau of Economic Analysis. Used by the Council of Chief State School Officers 
in its "State Indicators" reports. 

Developed by the Advisory Commission on Intergovernmental Relations and reported in its annual 
Significant Features of Fiscal Federalism series. Used by the Council of Chief State School Officers in 
its "State Indicators" reports. 
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Chapter 4 
School Process Statistics 



A. Rationale and policy questions - p. 66 

B. Current data collections and their limitations - p. 71 

C. Recommendations for improving collection and reporting - p. 79 



This chapter is based on an "idea paper" prepared for the National Forum by Dr. Andrew 
Porter of the University of Wisconsin at Madison. The draft manuscript for the chapter was 
written under the supervision of Dr. Martin Orland of the National Center for Education 
Statistics with contributions by Dr. Rolf Blank of the Council of Chief State School Officers, 
Dr. Anne Hafner of the National Center for Education Statistics, Dr. Joan Shoemaker of the 
Connecticut Department of Education, and Dr. David Stevenson of the U.S. Department of 
Education, Office of Research. It was prepared in cooperation with the Forum subcommittee on 
school process statistics chaired by Dr. Robert Burns of the Oregon Department of Education. 
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School Process Statistics 



Of the four data domains that constitute this national statistical agenda, recommending 
improvements in the school process area has proven to be the most challenging. The difficulty stems 
from two major sources. First, there is almost universal agreement that information about what and 
how much students are learning is essential for monitoring the quality of the Nation's schools. There 
is also general agreement that it is essential to monitor the Nation's resource inputs to education and 
the demographic characteristics of students. However, there is less-than-fuli agreement about the need 
for--and even the validity of-measures that address the nature of the schooling experience. These 
types of measures include school-process indicators such as who provides classroom instruction; what 
is being *aught, and how well; and what are the characteristics of the teaching and learning 
environment. 

Second, there is less precedent for national data collections in the area of school processes 
than in the three other domains. For example, there are no regularly collected national measures of 
instructional quality, chiefly because there is neither a consensus on what constitutes valid 
"indicators" of such quality nor agreement on how such indicators should he measured. Even efforts 
to collect seemingly straightforward measures such as teacher qualifications have been hampered by 
similar conceptual and technical concerns (e.g., how do you define a qualified teacher?). 

It is the view of the National Forum that despite these obstacles, school process measures- 
constitute a necessary and important component for monitoring the condition of education; 
informing national. State, and local policy decisions; and providing better accountability mechanisms. 

For the policymaker, there are three rationales for regular collection and reporting of sctuu>! 
process measures. First, process measures can describe instructional practice and, with this, the 
degree to which quality educational opportunities are available to all students in all schools. Do 
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children from poor families have the same opportunity in school to learn higher-order thinking ami 
problem wiving as do children from more affluent families? To what degree are students taking 
courses in high school that satisfy college entrance requirements? To what extent do teachers accept 
responsibility for siudent success or failure in learning? 

Second, process measures can monitor reform-xhn degree to whici* recommended changes in 
education practice are actually being implemented. Education in the Un:*cu states is periodically 
subject to reform efforts that often call tor substantial changes in current practice: changes in 
curriculum emphasis, changes in organizational structure, and changes in teaching techniques. While 
the motivation for reform is generally a concern lor student outcomes, the locus tor reform is 
typically on education practice. Monitoring these reforms requires a regular system of indicators on 
curriculum, on school environment, and on teaching. 

Finally, process measures can help to explain discrepancies in education performance or the 
possible reasons why student achievement may vary across locales and over time. For example, if 
student outcomes are improving more in one State than another, then knowledge of differences in 
curricula, instruction, and sehooi organization can provide policymakers with clues to explain these 
differences and point them toward promising future policy directions. While rect agnizing that 
relationships between schooling processes and outcomes are inherently complex, we believe that 
tentative and judicious explorations of such relationships by policymakers can lead to improved 
education policy and practice, 

A. Rationale and Important Policy Questions 
Key Principles and Assumptions 

The National Forum recommends that consideration be given to developing improved 
measures u* ' *chool-proccss indicators in three sub-domains: the implemented curriculum, teaching 
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quality in a subject area, and school environment. Within each sub-domain, our discussion and 
recommendations for statistical improvements are guided by the following five general principles: 

1 . To measure curriculum and teaching quality, the focus should be subject 
specific (e.g., math, science, reading); 

2. To develop process indicators, consideration should be given to their utility, 
feasibility, and burden; 

3. To address policy questions of interest with respectable validity and reliability, 
sample and universe surveys may be used even though other methods such as 
structured observations and interviews may provide richer detail; 

4. To monitor the relationship between process, outcomes, and equity, results 
should be analyzed separately by race/ethnicity, socioeconomic status, and sex 
wherever feasible; and 

5. To advance the art of measurement, special studies should he funded to pilot 
new techniques for measuring school processes. 

The following sections provide specific rationale? and associated policy questions for the three 

process sub-domains, along with discussions of current data collections and reports. Then, we provide 

a series of recommendations for data improvement in these sub-domains of school processes. 

1. The Implemented Curriculum 

Rationale. The "implemented'* curriculum refers to what is actually taught in classrooms: 
content and topic coverage, time and emphasis devoted to subject areas, course taking, and the 
context in which instruction occurs. It includes both broad subjects (e.g.. mathematics, history) and 
specific topic areas within these subjects (e.g., probability. American foreign policy) and is often 
measured by a time metric (e.g.. the number of minutes or class periods devoted to the subject i>. 
topic yea). 

The implemented curriculum is a key indicator because we cannot expect students to learn 
what they have not been taught, and we cannot take for granted thai schttols are organized in ways 
that guarantee that all students are exposed to important content. Fortunately research and common 
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sense appear to be in agreement on this matter. Researchers are finding teacher and classroom 
explanations for differential student achievement increasingly salient. While individual factors such as 
socioeconomic status and ability are still believed to account for a large proportion of variation in 
achievement, curriculum variables are often considered as important, if not more important, than 
home background and other student variables. 

A recent meta-analysis on variables related to learning, for example, found that quality and 
quantity of instruction were roughly equsl in importance to student characteristics and out-of-school 
contextual variables (Wang, Haertel, and Walberg. 1988). In particular, time on task (sometime:, 
referred to as "opportunity to learn") was found to be the most frequently cited variable in the 
instructional area. 

Similarly, Cooney and Dossey (1983) and Travers et al. (1985) argue that variables in the 
implemented curriculum are major culprits in explaining America's relatively poor education 
achievement in comparison with that of other nations. Thus, monitoring changes in student curricula 
exposure would appear to be critical to informing policymakers about whether our international 
achievement standing is likely to improve. Further, our commitment to equality of education 
opportunity would seem to require that policymakers and citizens have access to regular information 
on the instructional opportunities available to students from different backgrounds. 

Policy Questions. Given the fact that the national education goals recently adopted by the 
President and the Governors strongly emphasize academic preparation and performance, as well as 
high achievement in math and science, we believe that policymakers would find statistics on the 
implemented curriculum to be especially useful in addressing the following types of questions: 

1. What percentage of students currently enroll in various academic and vocational courses? 
How is this changing over time? 

2. Are increasing numbers of various groups of students enrolling in academic and 
vocational classes? 
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3. To what extent are schools offering academic and vocational courses? 

4. Within particular subject areas, what topics are covered? How much time and emphasis 
are allocated to topics? Are patterns of topic coverage changing over time? 

5. Do students from different backgrounds have equal access to and choice of different 
subjects and topic areas? 

2. Teaching Quality in Subject Area 

Rationale. While experts do not always agree on exact definitions and characteristics of 
teaching quality, there is an accumulation of research that identifies some indicators of quality and 
that validates their importance for enhancing student academic performance. In their sourcebook on 
education indicators, Shavelson, McDonnell, and Oakes (1989) maintain that the knowledge and skills 
of the teacher are important predictors of teaching quality. Their review of the research showed that 
academic knowledge and preparation in a teacher's subject area is related to student learning. This 
relationship varies by subject, with teacher preparation in science and math being particularly 
important. 

At the national and State levels, the academic and professional preparation of teachers and 
teacher certification for a specific teaching assignment are important pol icy-relevant indicators of the 
degree to which teachers in a subject area meet minimum State requirements. The policy interest 
here is the match between preparation and certification of teachers am! their teaching assignments. 
Data on teacher preparation and certification also provide a starting point for measuring teacher 
shortages and demand by subject (e.g.. the extent of the shortage of certified science teachers in New 
York). 

Finally, Porter (1989) notes several characteristics of good teaching, many of which appear 
amenable to regular and systematic measurement. For example, good teaching makes efficient use of 
student time and is carefully constructed to enhance the possibility that students actively participate in 
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the learning process rather than passively absorb what they are told. The amount of time spent in 
lecture and independent seatwork is kept in balance by instructional strategies that provide for student 
discussion and cooperative, team, and hands-on activities. 

Effective teachers also clarify for students what is to be learned, and why, and translate 
subject matter and pedagogical knowledge into conditions that enhance student learning. Such 
teachers also accept appropriate responsibility for student success or failure in achieving desired 
outcomes. 

Policy Questions. These research findings suggest that policymakers would be interested in 
data that address the following questions related to the quality of teaching: 

1 . What is the academic and professional preparation of teachers assigned to a 
given subject area? 

2. What proportion of teachers assigned to a subject or field are not certified in 
that subject or field? 

3. What types of students are exposed to teachers who are assigned out of field? What 
courses do such teachers instruct? 

4. How do teachers allocate class time 9 

5. To what extent do teachers employ appropriate instructional strategies that engage students 
in active learning? 

6. To what extent do teachers accept responsibility for student success and failure 
in achieving desired outcomes? 

3. School Environment 

Rationale. The literature on planned change, staff development, and effective schools 
suggests that those in the best position to improve schools are the staff in the schools. A productive 
environment promotes and nurtures collaborative decisionmaking and staff involvement in the 
planning and evaluation of school programs. A selfdirected school environment leads to greater staff 
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responsibility for successful teaching and increased learning, and it also creates a climate for school 
changes to occur. 

Moreover, schools engaged in change and improvement processes are likely to be more 
successful in creating an environment conducive to higher academic achievement. Schools committed 
to student learning are characterized by high teacher expectations for student success, student support 
for academics, a large number of academic courses available, and systems for monitoring and 
rewarding academic achievement. 

Policy Questions. High priority school-level policy indicators in this sub-domain are 
recommended in three specific areas: academic emphasis, conduct veness to learning, and the 
decisionmaking environment. We expect that such measures would be useful to policymakers to help 
answer the following types of questions: 

1 . Is there a school-wide focus on academic achievement for all students (school academic 
emphasis)? 

2. What are the course requirements for graduation? 

3. To what degree are schools safe, disciplined, and drug-free? 

4. To what degree do schools have a shared decisionmaking process on curriculum and 
instruction issues? 

B« Current National Data Collections: Their Limitations and 
Potential Strategies for Improvement 

1, Implemented Curriculum 

Until recently, the NCES longitudinal studies (National Longitudinal Study-1972 |NLSJ. High 
School and Beyord. and the National Education Longitudinal Study: 1988 INELS:881). the National 
Assessment of Educational Progress (NAEP) Transcript Study, and the International Association for 
the Evaluation of Educational Achievement (IEA) studies were the only sources of national data on 
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the implemented curriculum. These surveys include data on courses in which students are enrolled. 
Class type (track) data have been collected, for example, by the IEA studies (Educational Testing 
Service 1987, NCES 1985) as well as by the NCES longitudinal studies (NCES 1981a-b and 1989a-b) 
and the NAEP transcript studies (Educational Testing Service 1989). Longitudinal studies, in 
particular, permit comparisons to bt made over time (1972, 1980, 1957. and 1990) and among 
population subgroups on such questions as the percentage of students enrolling in different academic 
and vocational courses. In addition, a national school "Offerings and Enrollments Survey" was 
conducted in 1982 through High School and Beyond (NCES 1982), and another is planned for 
NELS.88 in 1992 (NCES 1989e). Although these national studies are of high quality, they do not 
provide State-level estimates. 

Inf ormation on specific topic coverage within subject areas has been available from the 
international IEA and the NCES longitudinal studies. In the IEA math and science studies, teachers 
were asked whether students in their classes have had the opportunity to learn the content of a series 
of items, about the perceived degree of newness of content topics, about methods used in teaching 
content, and about the reasons for using or not using various interpretations of content. 

NELS:88 "Base-Year* (8th grade) and "First Followup" (10th grade) teacher questionnaires 
include some items on topic coverage and emphasis for mathematics, science, reading, and social 
studies (NCES I989b-c). For example, in 8th-grade math, topics include categories such as fractions, 
ratios and proportions, and geometry. 

In 1988. NCES initiated the Schools and Starting Survey (SASSK which provides national and 
State-level estimates of the time allocated to subject-area coverage at the elementary grade levels 
(NCES 19890. This survey is to be repeated in 1991 and every two years thereafter; thus, trends on 
this dimension could be regularly monitored. In addition, the Council of Chief State School Officers 
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(CCSSO) has recently begun collecting State-level administrative data on math and science cov*se- 
taking patterns at the secondary level (Council of Chief State School Officers 1989). 19 

These two new data sources attempt to address the primary weakness we have identified in the 
current collection and reporting system in this area: the absence of comparable, high-quality 
State-level data on student exposure to specific academic and vocational courses. The need for such 
data is particularly acute at the secondary levd. While the current CCSSO project represents a useful 
first step in addressing this need, it also serves to illustrate the considerable difficulties that must be 
overcome in developing such a measure. State-level aggregate data covering the areas of math and 
science are not available from some States, and among participating States, substantial efforts to 
"crosswalk" State records to a common reporting format were often necessary. NCES, the CCSSO. 
and the States need to continue to work cooperatively to develop a common system for collecting and 
regularly reporting accurate and comparable data on student course-taking. NCES also needs to 
examine the reliability and usefulness of estimates derived from the SASS collection on the time 
teachers report they allocate to different subject areas in the elementary grades. 

Another weakness in the current data system is the absence of detailed information on topic- 
area coverage (e.g., weeks or class reriods spent on topics) as well as specific instructional 
techniques. As noted earlier, the NELS:88 survey does include topic coverage items. However, the 
variables tend to be rather broad, and the time spent on topics is not included. 

Creating valid and reliable measures at the topic-area level is critically important because, as 
research has consistently shown, similarly labelled courses (such as "Beginning Algebra") often cover 
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substantially different content elements. Furthermore, the movement toward interdisciplinary course 
offerings may render traditional broad curriculum descriptors even less useful in the future. 30 

Given the status of current national collections and the statistical "state of the art/ the short- 
term data improvement objective with respect to measures of specific topics covered would not be to 
produce State-level estimates. Rather the goal would be to regularly report currently available, 
relevant national-level statistics describing student "opportunities to learn" different topics, as well as 
how these opportunities might differ for different types of students and how they are changing over 
time. In the longer term, the development of better measures of curriculum topic areas and their 
breadth and depth of coverage would be useful. This will require considerable new research as well 
as validation studies that pilot the newly developed measures. Ultimately, the objective would be to 
regularly report such data at both the national and State levels. The recent identification of national 
education goals may point to specific curriculum indicators on which initial attention should be 
focused and at what grades. For instance, the goal of enhanced student achievement for grades 4, 8. 
and 12 in English, mathematics, science, history, and geography suggests initial development of 
content coverage measures and allied indicators in these areas and grades. 

2. Teaching Quality in Subject Area 

In general, data from State administrative records systems and cross-sectional surveys are 
potentially available to address questions regarding basic teacher qualifications, while information on 
classroom instructional practices can be obtained from NAEP and from NCES longitudinal surveys. 
The previously noted CCSSO science-math indicators project reports State-by-State data on teacher 



* This is not meant to argue against the importance of collecting and reporting information at the broad 
"content domain" level. It is to recognize, however, the inherent limitations of such data, and therefore, 
the need for more detailed topic area collections in order to provide valid and reliable measures on such 
salient topics as student "opportunity to learn." 
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assignments by certification status in six science fields, mathematics, ami computer science tor the 
1988-1989 school year (Council of Chief State School Officers 1989). The data collection and 
reporting will be repeated for the 1989-1990 school year, and CCSSO expects 40 States to report 
these data frc>m their teacher personnel files and certification records. Because high school enrollment 
data are also collected in this study for specified mathematics and science subjects, the opportunity 
exists in the participating States to relate trends in teacher certification with subject area enrollments. 

In this study, out-of-field teachers are divided into two categories-teachers assigned more 
than 50 percent of the time to a subject/field (primary assignment) and teachers assigned less than 50 
percent of the time to a subject/field (secondary or other assignment). Ideally, an ITE statistic could 
be used to generate one figure, but about half the States do not collect teacher assignments by period 
or lime. 

The NCES Schools and Staffing Survey collects data from representative samples of teachers, 
schools, and districts (NCES I989f). A major purpose of SASS is to provide periodic indicators of 
teacher supply and demand. The teacher survey is designed to collect 3nd report data at the national 
and State levels on the numbers and characteristics of teachers, including specific teaching 
assignments and teacher preparation (degrees, experience, course credits, staff development). The 
data include assignment by certification and teachers' self-reports on whether they are qualified in 
their fields of teaching assignments. 

At the national level, these data are quite complete. However, the State samples in SASS are 
not sufficiently large in many States to produce valid statistics on teacher supply patterns for 
particular teaching fields such as biology, physics, and computer science. Thus. SASS would not be 
able produce State-level estimates of status and trends regarding teaching qualifications for specific 
subjects. It could, however, provide State estimates within broad subject areas such as mathematics 
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and science and (to examine distributional patterns within States) link this information with school 
background characteristics such as percent minority and percent eligible for free lunch. 

NCES* Recent College Graduates (RCG) Survey is also available to trace national trends on 
the backgrounds and qualifications of new entrants into the teaching profession (NCES 1989d and 
1990a). This survey collects data at two-year intervals from a nationally representative sample of 
college students one year after graduation, and it reports statistics on new-teacher preparation, 
certification, and teaching assignments. 

NELS88 includes surveys of teachers and principals, along with student and parent 
questionnaires (NCES 1990c). One of the major purposes of NELS:88 is to analyze teaching quality 
in subject areas and its relationship to student background characteristics and outcomes. Measures of 
teaching quality include questions on teacher preparation (degrees, certification status, experience, 
course credits, and preservice and inservice staff development and training), assignments, teaching 
strategies and techniques, time allocation, workload, teacher efficacy, and attribution of responsibility 
for student outcomes. NELS:88 will provide national statistics on teaching in 8th-. 10th-, and 12th- 
grade classes, but it cannot at present provide State-level estimates (with the exception of the 1 1 States 
that have either "bought into" NELS:88 to obtain a State-representative sample or have large enough 
representation in the national sample). 

NAEP assessments generally include teacher questionnaires that ask about broad teaching 
strategies and time allocation by subject (Johnson and Zwick 1990). However, the level of detail on 
this topic is limited (compared with NELS) and the data are not consistently analyzed and reported. 

With the exception of the intermittent IEA studies, no national survey has previously 
measured content-specific instructional techniques, and most likely some development work is needed 
in this area. In the IEA Mathematics study, teachers were asked to give data on sources of 
instructional materials, methods used in teaching content, and reasons tor using or not using various 

76 

rs 



interpretations of content (NCES 1985). In addition, topic-specific questionnaires were administered, 
focusing on auricular and cognitive processes on a specific topic along with number of methods and 
time allocations (e.g., for algebra, fractions). 

Considering the data elements that are currently available and the need for policy-relevant 
indicators of teaching quality, top priority in the short run should be placed on collecting and 
reporting national- and Sta^-level data on the number of teachers who appear to lack the minimum 
qualifications necessary to teach in their assigned field. Reliable national estimates of this can be 
obtained from SASS (based on teacher self-reports of certification) and should be collected and 
reported with regularity to inform this question and note trends over time. 

Uniform and valid State-level data on teacher certification by subject field are not currently 
available but could conceivably be collected by enlarging the SASS survey sample considerably or by 
drawing upon State data files (universe of teachers) of certificated staff by field, as demonstrated in 
the CCSSO science-math indicators project. The former option, while far more costly, would provide 
richer measures of teacher preparation levels, including-in addition to certification status-college 
major, courses taken in the area being taught, and self assessment of qualifications u> teach in that 
area. :i 

NELS:88 may demonstrate that national data on such topics as specific instructional strategies 
and techniques, teacher workload, teacher acceptance of responsibility lor student success, how 
teachers are assessed, the amount of inservice training teachers receive, and allocation of time by 
teachers are important, valid, and reliable "stand-alone" policy indicators of teaching quality. That is. 
NELS:88 may show that particular practices in these areas are clearly and consistently associated with 



;| SASS can (and should) l»e used to provide National-level estimates of these conditions by subject field 
on a regular basis. 
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higher student outcomes. To the extent that this occurs, periodic data collection at State and national 
levels should be considered through surveys such as SASS. 

3. School Environment 

NCES currently collects substantial data on school environment in both its longitudinal and 
cross-sectional surveys. High School and Beyond and NELS:88 can provide national data on school 
academic emphasis, school size and structure, curriculum offerings, discipline, staff development, and 
the availability and use of "high technology" materials in the classroom (e.g.. personal computers and 
calculators). In addition. NELS:88 "First Followup," which will be tested in 1990. has data elements 
in the following areas: shared decisionmaking, teacher and principal autonomy, principal leadership, 
and student drug and alcohol use (NCES 1989c). To the extent feasible, comparisons should be made 
between the schools serving the NELS lOth-grade cohort and those serving the High School and 
Beyond lOth-graders of a decade earlier on these dimensions. 

The School Effects supplement to NELS:88 will be carried out from 1990 to 1992 at the 10th- 
and 12th-grade levels in about 270 urban schools {NCES 1990b). The student and teacher samples 
will be augmented to make it possible to study which aspects of sehools and teachers are most 
"effective" in enhancing student achievement. The primary policy focus is on identifying school and 
department organizational and management characteristics, and teacher practices and beliefs that relate 
to higher achievement and other positive student outcomes, especially with pin>r and at-risk 
populations. The Sehools and Stalling Survey collects data on teacher satisfaction and commitment, 
school policies, working conditions, student drug and alcohol use. school discipline and violence, 
grouping and tracking practices, and staff problems (NCES 19890. Since SASS is a cross-sectional 
study of schools and does not contain outcome measures, it is impossible to link these school 
environment variables to changes over time in school effectiveness. However, SASS can provide 
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"snapshots" or gross-level indicators at the national and State levels of the environmem in our 
Nation's schools every two years and can relate these data directly to student and community 
background characteristics. 

The need for measures of drug and alcohol use and crime in the schools has taken on added 
relevance since the joint declaration of national education goals by the Nation's Governors and the 
President. As noted, some information from school staff on drug and alcohol use and crime and 
violence levels can be obtained every two years by NCES through the SASS at both the national and 
State levels. To obtain additional information on such topics as school policies toward students found 
to possess or distribute drugs or alcohol, and school curricula and instruction on these matters, would 
require either additional items on the SASS or the development and administration of a new survey. 
NCES* Fast Response Survey System (FRSS) could be used for this purpose. The FRSS allows for 
the administration of short (one-page) surveys to a representative sample of schools on a discrete topic 
area (NCES 1990b). Another strategy would be for NCES to work cooperatively with other agencies 
concerned with these issues (e.g.. Centers for Disease Control, U.S. Drug Enforcement Agency. 
Department of Justice) to collect and report the necessary measures. 

C. Summary of Recommendations for Improving the Collection and 
Reporting of School Process Statistics 

Although our discussion of school process variables was divided into three sub-domains 
(implemented curriculum, teacher quality, and school environment), we are struck by the 
interrelatedness of the areas. Together they describe the context for education. Access to knowledge 
is provided by the implemented curriculum, but the quality of that curriculum is related to the quality 
of instruction and the professional conditions for teaching. And both curricular and instructional 
quality are dependent upon a school environment that has the capacity for continued renewal and that 
supports a press for academic achievement. 
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Just as th<» process sub-domains are interrelated, so too are the data elements. For example, 
information about patterns of course taking in science and mathematics not only describes the 
implemented curriculum, but it can also be used to describe the degree of academic emphasis in the 
school environment. Information about teaching quality, such as the backgrounds of teachers who 
teach particular kinds of courses, can also be used to describe equality of opportunity for all students 
to learn important content. 

Because school process data make up the least developed domain in the current national 
education statistical system, it is evident that implementing the improvement recommendations 
outlined below will frequently require additional research to create conceptually valid measures. It 
will also require careful consideration of the feasibility, costs, and burdens associated with alternative 
data collection strategies* More precise definitions of desired data elements, modes of collection, and 
implementation timeliness must, of course, await the results of this work. Thus, as in the other 
domains, these recommendations represent broad directions for change that are intended to guide 
future data improvement work. 

The National Forum makes the following six recommendations for improving data collection 
and reporting in the domain of school process statistics: 

1 . The National Center for Education Statistics (NCES) should regularly collect and 

report national and comparable State-level data on student enrollment in academic and 
vocational secondary courses by race/ethnicity, sex, and other demographic subgroups 
as feasible and appropriate. To accomplish this, NCES musi first develop procedures 
for ensuring the collection of broadly comparable data across States on secondary- 
school course offerings. The Office uf Educational Research and Improvement 
(OERI) should also determine the usefulness of collecting State-level data on time 
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allocated to subjects in the elementary grades (such as that currently collected in the 
Schools and Staffing Survey (SASS1). 

NCES should regularly collect and report data at the national level on broad indicators 
of teacher preparation (e.g., certification status, number of courses taken in teaching 
area, major field, and preservice and inservice development and training experiences) 
by specific teaching assignment. Trends on these measures should be related directly 
to changes in the size of the teacher work force as well as student enrollment patterns 
(i.e.. teacher supply and demand). In addition. NCES should investigate the 
feasibility of regularly collecting and reporting comparable Srate-by-State statistics 
using such measures and of reporting on the numbers of new teachers certified via 
"alternative" routes. 

NCES should regularly collect and report data at the national level on student 
"opportunities to learn" specific instructional topics. Work should begin first on the 
high-priority subjects included in the national education goals (English, mathematics, 
science, history, and geography) and then proceed to other subjects. OERI should 
develop new measures of the depth and breadth of coverage for these topics for 
possible future collection and reporting at the national and State levels. 

NCES should regularly collect and report nationally representative data on the school 
environment including school-level measures of academic emphasis (e.g.. curricular 
offerings and enrollments) and decisionmaking practices. To the extent feasible. 
NCES should relate such data to important background characteristics of students 
attending these schools (e.g.. sex. race/ethnicity, handicapping condition. 



socioeconomic status) as well as key demographic characteristics of the larger school 
community. 

In order to measure progress in meeting the national goal of "safe, disciplined, and 
drug-free schools" (goal No. 6 adopted by the Nation's Governors and the President), 
NCES ot other Federal agencies should regularly collect and report national- and 
State-level data on drug and alcohol use and on violence in the schools, as well as on 
policies and programs undertaken to prevent such occurrences. To develop measures 
of these, NCES should proceed immediately to examine the feasibility of augmenting 
its current sample surveys (e.g., SASS). mounting a new survey (e.g.. using the Fast 
Response Survey System), or working in concert with other agencies concerned with 
these issues (e.g.. Centers for Disease Control, Drug Enforcement Agency). To the 
extent feasible, these data should be related to the background characteristics of 
students and their home communities. 

OER1 should fund special studies to improve the me iment of important school 
processes including academic emphasis, subject-specific instructional strategies, depth 
and breadth of content coverage, the use of new technologies in instructional programs 
(e.g., personal computers), and methods of training teachers and assessing their 
competence. Newly developed measures created through such special studies may 
eventually be incorporated into future regular national collections and reports. 
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Student Outcome Statistics 



Prior to the 1980s, parents, legislators, Governors, and leaders of business and industry 
frequently asked the question, "How are our education dollars being spent?" Today, these same 
people appear more likely to ask, "What is the result of spending our education dollars?" The 
Nation's citizens and policymakers increasingly demand information about the resutts-the outcomes- 
of schooling. They want to know how students are progressing through the system, what students are 
learning and achieving, and whether students are being adequately prepared to meet the labor-market 
demands of an increasingly sophisticated and global economy. 

A. Rationale and Important Policy Questions 

The information being asked by policymakers about education outcomes for students is 
reflected in the following types of questions: 

• What do our students know? Do they know as much as students in other States and 
countries? 

• How many of our students complete high school? How many drop out? How do our 
graduation and dropout rates compare with those of other States and the Nation as a 
whole? 

• What do students do after high school? How many attend postsecondary institutions? 
How many enter the military? How many enter the job market? 

• Are achievement levels, completion rates, and the postsecondary-education enrollment 
and employment status of our students improving, staying the same, or declining over 
time? 

These questions reflect our Nation's growing interest in what students learn throughout their 
K-12 education and how prepared they are for the transition to postsecondary education, employment, 
and adulthood as responsible and productive citizens. They also illustrate the need tor accurate 
infWmation that policymakers can use in making decisions about allocating new education resources 
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of reallocating existing ones; continuing current programs or developing new ones; and developing or 
revising policies, rules, and regulations. 

In September 1989 at the education summit in Charlottesville, Va., the President and the 
Nation's Governors began the process of identifying national goals for education. In February 1990. 
they announced a set of goals that include dramatic improvements in academic achievement and high 
school graduation rates over the next decade. Many States have also established or are in the process 
of establishing education goals of their own that include raising student achievement levels, school 
completion rates, and postsecondary education achievements and status. Establishing and reporting on 
national and Statewide outcome measures can provide the States and the Nation with indicators of 
progress toward these goals. 

Because States have the primary responsibility for education, it is important that they be able 
to assess and compare progress toward important national goals, such as those developed by the 
Governors and the President. Valid, comparable student outcome measures will improve public 
understanding of the condition of education and may help mobilize public interest in and support for 
the Nation's schools. Conversely, the inappropriate collection and reporting of such measures may 
result in data that are not truly "comparable" or reflective of how well our schools are doing and how 
much our students are achieving. 

Key Principles and Assumptions 

In reviewing the current status and adequacy of national statistical collections and reports on 
student outcomes, we have been guided by the following general principles and assumptions: 
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1. Outcome measures alone are not sufficient to answer questions about why 
things are the way they are. 

Such measures are more likely to be descriptive than diagnostic. They can be likened to a 

speedometer-it can tell you how fast or slow you are going but not why. If the "pedal is to the 

metal" and the speedometer shows only 20 miles per hour, you know something is wrong. 

Determining what is wrong requires additional information that the speedometer is not likely to 

provide. 

2. Wherever possible, existing measures and data collection instruments should he used. 
In general, improving current and existing instruments and systems to provide additional 

information or to provide comparable data at the State and national levels should take priority over 
development of another set of instruments or systems. One strategy for providing additional useful 
outcome data is to expand the sample of existing national -level surveys to permit reliable State 
estimates in areas of policy interest. States should have the opportunity to "piggy-back" on these 
samples and gather data on other issues if they w ish. 

3. Improvements should focus on better coordination of existing national surveys. 
Survey samples and data collection from, and linkage of data on. elementary, secondary, and 

postsecondary education institutions by the National Center for Education Statistics (NCES) through 
the Common Core of Data (CCD), the Integrated Postsecondary Education Data System, the National 
Assessment of Educational Progress, and the longitudinal studies (High Sehiu>l and Beyond, National 
Education Longitudinal Study JNELSl, and National Postsecondary Student Aid Study 1NPSASJ) 
could result in less duplication of effort: less burden on individual States, districts, and schools; better 
measures of outcomes; and better understanding of education processes. 



4. In the short term, sampling offers the quickest and feast burdensome 
method of obtaining data on student achievement that is comparable 
across States. 

This principle is based on the assumptions that there will be greater coordination in sampling 
and that individual State samples will be large enough to make State-by-State comparisons. 

5. Reports should be generated on student outcomes to show the following: 

a. comparisons of States and the Nation with a criterion standard; 

b. trends over time (if this does not restrict improving assessment items and use of 
innovative testing and measurement techniques); 

c. State comparisons with national norms and the norms of other States; and 

d. comparisons among important subgroups (sex, race/ethnicity, economic status, and 
language status) at both ihe national and State levels. 

B. Current National Data Collections; Their Limitations and 
Potential Strategies for Improvement 

The ensuing discussion of current national data on student outcomes and their limitations 
addresses four types of student outcomes: 

• Student achievement; 

• Student participation and progression; 

• Student status after completion of secondary school; and 

• Student attitudes and aspirations. 

For each we reiterate the policy questions that should be informed by such data, point out the 
status and limitations of current statistical collections and reports, and recommend specific 
improvements in the current system. 
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1. Student Achievement 

The regular collection and reporting of student achievement data are necessary to tell us the 
degree to which students are meeting established education objectives. There are several national 
collections and reports that provide information on student achievement. 

The National Assessment of Educational Progress (NAEP) has been assessing student 
achievement since 1969. NAEP has several features that make it an ideal system for assessing and 
monitoring student achievement: 

• NAEP reports achievement of 4th. 8th, and 12th grade students on a regular schedule 
in the areas of reading, mathematics, writing, social studies, geography, and science. 
This structure makes it possible to report on trends in student achievement over time. 

• The grade levels at which testing is done reflect points of transition tgrade 4. 
the transition to more complex reading comprehension and arithmetic 
operations and understanding; grade 8, the conclusion of arithmetic instruction 
and transition to college preparatory work in mathematics and other areas; and 
grade 12. the conclusion of secondary schooling). 

• Proficiency levels have been established for reading, writing, mathematics, and 
science. Thus, meaningful statements about student achievement in meeting 
specific curricula objectives can be made (Johnson and Zwick 1990. Educational 
Testing Service 1989b). 

Although NAEP provides a regularly reported national measure of student achievement based 
on a nationwide sample of students, its principal limitation is that comparisons of student achievement 
in the States cannot be made because the sample is not representative of individual States. As a 
result, policymakers are forced to turn to other measures of student achievement for State data- 
measures with severe limitations. 

Data on Scholastic Aptitude Test (SAT) and American College Testing Program (ACT) 
performance are the most prevalent student achievement indicators used nationally and by individual 
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States in reporting State-level outcomes. 22 Student scores on Advanced Placement tests administered 
by the College Board are sometimes also used for State-level reporting (Office of Planning. Budget, 
and Evaluation 1990). The Armed Services Vocational Aptitude Battery (ASVAB) is a data base that 
could be used to compare achievement of students who do not enroll in college immediately after they 
graduate from high school. National and State summaries are available for each of these tests. 

There are two major reasons why the use of any of these measures to assess student 
achievement at the State level is problematic. First, the population of test takers is. to a large degree, 
self-selected and unstable over time. Test takers are not comparable from State to State, and they 
may not be representative of the total population to be assessed. In addition, changes in the population 
of test takers over time in a State make even within-State comparisons over time problematic. 
Second, the content coverage of these measures is limited. This reflects the fact that they were 
expressly designed for purposes other than comprehensively assessing student achievement. 

These problems seriously compromise the usefulness of such measures to guide performance 
and policy development. However, their widespread use. despite the known limitations, clearly 
illustrates the importance policymakers attach to regularly reported State-level student achievement 
data. Thus, in the absence of more appropriate instruments and mechanisms, such measures will 
continue to be used to gauge State-level student performance. 

A pilot project involving 41 States to determine the feasibility of a "State NAEP" holds some 
promise of providing more comparable State and national data on student achievement.- 1 If the pilot 
proves successful. NAEP should be used to report regularly on student achievement in the States in 



~ In addition to their use by several States in monitoring trends in student achievement, these measures 
have also been reported annually by the U.S. Department of Education in the State Education 
Performance Chart (Office of Planning, Budget, and Evaluation 1990). 



^ A discussion of critical design considerations that should be addressed in the successful implements 
of State-level NAEP assessments can be found in a recent technical report by NCES (NCES I989e). 
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core subject areas. The State samples should also be large enough to permit analyses of the 
performance of important subgroups (e.g.. racial/ethnic groups. language-minority groups) and 
different types of education units (e.g.. school districts with high/low poverty concentrations) within a 
State. While the cost of such expansions would be high, the cost is likely to be less than it would be 
to design and develop a completely new system. 

Our general endorsement of NAEP (or an equivalent type of national assessment instrument 
with comparable features) and our recommendations for its expansion to the State level do not mean 
that we wish the general structure of NAEP assessments to remain unchanged. In recent years, the 
assessment field has turned from "paper and pencil" multiple-choice assessment techniques to broader 
and more "hands on" measures of student learning, sometimes called performance assessments. 
NAEP (or its equivalent ) should take a leadership role in employing such measurement techniques in 
its student achievement surveys, and the Federal Government should vigorously pursue research, 
development, and experimentation with these measures. 

In addition, it is essential to have periodic international assessments of student performance, 
comparing U.S. students with those of other industrial ized nations (Educational Testing Service 
1989a). Only in this way will we know whether our student achievement levels are meeting an 
international performance standard. The recently endorsed national goal of improving the 
international performance of U.S. students in science and mathematics achievement demonstrates the 
need for this kind of measure/ 4 

Finally, although our principal concern is with outcome measures as "stand alone" indicators 
of the performance of the education system, we also believe that as "The Nation's Report Card," 
NAEP could help us better understand the reasons for education outcomes. To do this. NAEP 



" 4 That goal, endorsed by the Nation's Governors and the President, states: "By the year 2000. U.S. 
students will be first in the world in science and mathematics achievement." 
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achievement results need to be linked with data on student course-taking and student and schooling 
environments. This could be achieved by collecting additional data through NAEP or by linking 
NAEP and other national surveys 25 . One strategy that deserves serious consideration by NCES is to 
link NAEP with NELS by equating items from the two assessment instruments. NELS is an 
extremely rich source of data on student background characteristics and instructional processes. 
Building a bridge between these data and NAEP results could produce a powerful tool for analysts to 
use in unraveling the complex relationships among student characteristics, schooling processes, and 
student achievement. 

2. Progression Through the Education System 

Information on student progress through the education system can inform policymakers about 
the percentages of students who drop out and complete school and the extent to which these 
percentages are changing over time. 

NCES currently collects State data on numbers of high sehi>ol completers through the CCD 
surveys (NCES 1990). Such data have been used in the past to calculate high school completion rates 
(in addition, dropout rates are often inferred from completion rates). Stale and national graduation 
rates are reported in Education Department publications such as the Condition of Education (NCES 
19890, the Digest of Education Statistics (NCES 1989a), and the State Education Performance Cliart 
(Office of Planning. Budget, and Evaluation 1990). The principal weaknesses of the current data on 
school completion are the lack of comparability and completeness in the data reported by States and 
the lack of information to determine completion rates for important State subgroups, such as 
minorities. 



25 See Baron et al. (1986). Jones (1986) and Spencer (1986) for discussions of technical and substantiv 
issues associated with linkages of this type. 
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Dropout rales are currently estimated and reported annually by the Bureau of the Census 
Current Population Survey (Bureau of the Census 1988) as well as in periodic special stupes such as 
the NCES National Household Education Survey (NCES 1989d). However, individual State estimates 
are not available from these data sources. 

Obtaining State-level measures of school dropouts and completers requires some way of 
tracking students as they move through the system. This can be accomplished through administrative 
recordkeeping or longitudinal studies of a student sample. Regardless of how the data are obtained, 
they must be based on common definitions of "dropout" and "program completion" (NCES 1989b). 
Both methods also require identifying data elements to be collected so that the characteristics of 
dropouts and program completers can be identified. At a minimum, such descriptive data elements 
should include sex and race/ethnicity. 

NCES longitudinal studies such as High School and Beyond and NELS are potential sources 
of information on State-level completion and dropout rates. 2 * However. State-by-State data and 
comparisons generally are not available from these sources because the State sample sizes are tin* 
small. 27 In addition, because new cohorts for these surveys are begun relatively infrequently 
(approximately once every eight years), their usefulness in monitoring trends over time is restricted. 
While studies of this type could theoretically be conducted more frequently or expanded to obtain 
representative samples of States, the costs would be substantial. 

The advantage of using administrative records to report dropout and completion statistics is 
that once a system of recordkeeping and reporting is developed, it could be used to report comparable 
State-by-State information (as well as comprehensive within-State data) on an annual basis. Ideally. 

* The High School and Beyond Survey has previously been analyzed by NCES fos this purpose (NCES 
1986). 

21 Eleven States have either supplemented the NELS sample or their samples contain enough observations 
to provide State representative information. 
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each State would have a Statewide data base or student information system capable of tracking 
students as they move from school to school within the State. While few States currently have such 
systems, the technology is rapidly becoming available. Now is an opportune time to begin to identify 
the data elements and other information that should be common to all student records. 

NCES is conducting dropout statistics field tests in 27 States to determine whether it is 
possible to obtain comparable State-by-State dropout rates by expanding administrative-record 
reporting through the CCD process (NCES 1989e). M If the field test determines that this approach 
is feasible, it could be implemented nationwide and the results used to report both annual and 
cumulative (i.e., grades 9-12) dropout rates each year.^ 

Given the recently endorsed national goal on high school completion.*' it is especially 
important that we regularly collect and report accurate State-level data on the percentage of high 
school completers. The Council of Chief State School Officers* Education Data Improvement Project 
(EDIP) has recently identified variations in how States report such data to the CCD (Clements 1990). 
Relying on the work of a Task Force convened to discuss where reporting variations exist and to 
make recommendations for collecting more comparable and complete data. EDIP issued a report tint 



28 In this field test, an annual dropout is defined uniformly as a student (1) who was enrolled during the 
previous school year but was not enrolled at the beginning of the current year; (2) who has not graduated 
from high school or completed a State- or district-approved program; (3) who has not transferred to 
another public school district, private school, or State- or district-approved program; and (4) and who 
has not been suspended, expelled, or excused from school due to illness (and who has not died). 

w The 4-year cumulative dropout rate could be defined as the percentage of 9th-grade students within 
a State who fail to graduate from high school or complete an alternative certification program or 
examination and who are no longer enrolled in school for reasons other than death or extended illness. 
This rate could be estimated using a "synthetic cohort" by aggregating, for a given year, the percentage 
of such students dropping out of school in each grade (grades 9-12). 

" The goal stipulates that by the year 2000. the percentage of students graduating from high school will 
increase to at least 90 percent. 
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explicitly defined tour mutually exclusive categories of "high school completer."' 1 The Task Force 
recommended that NCES collect and report data annually from each State using these school 
completer categories and that NCES periodically examine and report long-term high school 
completion rates (through age 24) by State. 

3. Student Status After High School 

Data on the status of students after completing high school provides the following information 
on what happens to recent high school graduates. 

• how many enroll in postsecondary institutions; 

• how many enter the military; and 

• how many enter the labor force. 

Potential sources of data in each of these areas are the NCES longitudinal studies. As with 
completion and dropout data, the current samples are generally too small to provide reliable State- 
level estimates. However, NCES should continue to offer States the option of supplementing, at their 
own expense, national longitudinal studies to generate State-level reports. 

A complementary strategy is tor NCES to investigate whether the desired data could be 
obtained through better articulation among its own elementary -secondary and postsecondary surveys 
and between each of these surveys and other national data collections. 



M The categories are: 

• the number and percent of students receiving regular high school diplomas through 
traditional programs; 

• the number and percent of students receiving regular high school diplomas through 
nontraditional programs; 

• the number and percent of students receiving certificates of attendance or completion; 

• the number and percent of students receiving credentials based on passing the GED tests. 
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One avenue to sxplore in the area of postsecondary attendance rates is the feasibility of 
linking the CCD collection of high school graduates by State with the NCES Integrated Postseconda 1 . ■ 
Education Data System (IPEDS) collection (NCES 1987 and 1988). It may be possible to aggregate 
totals from the IPEDS survey by State of origin (as well as race/ethnicity and sex, within States) and 
then combine them with impioved CCD counts of high school completers to create a State "college- 
going rate" statistic. The goal would be to obtain, at least every two years, estimates of the 
percentage of each States' high school graduates entering different types of postsecondary institutions 
(two- and four-year colleges, trade schools, etc.) within 12 months of graduation.' 7 

Another option in the area of postsecondary education attendance patterns would be for each 
State to develop its own method for obtaining first-time enrollment data from post-secondary 
education institutions and then to report its findings directly to the CCD. Regardless of the strategy 
employed, the number of first-time enrollees who graduated from high schools from each State in the 
preceti.ng 12 months would provide an important measure of the flow of students from secondary to 
postsecondary education. These data could also be used to monitor s**ident migration from one State 
to another in terms of postsecondary attendance. 

Information on the military enlistment rate of State high school graduates could potentially be 
obtained from Department of Defense data, analyzed in conjunction with improved CCD graduate 
counts. If the enlistment data could be disaggregated by State and reported by race/ethnicity, sex, 
and age. the resulting measures would indicate, for each of these categories, the percentage of high 
school completers enlisting in the military within 12 months of graduation. 



33 Ideally, the percentage of a States' high school graduates who attend postsecondary institutions at any 
time would be good information to have. However, obtaining that information would require extensive 
followup and high levels of cooperation between postsecondary institutions and State education agencies, 
and it would be expensive. It is, thus, not considered a high priority statistical improvement 
recommendation. 
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Through Current Population Surveys ami the Bureau of Labor Statistics, it is possible to 
obtain national, regional, and, in some cases. State-level statistics on the employment conditions and 
status of various age cohorts (Bureau of the Census 1990, Bureau of Labor Statistics 1989). It may 
be possible to coordinate NCES data collections with these surveys in order to determine the 
percentage of a States* recent high school graduates who enter the civilian labor force and are 
gainfully employed. 

ShouM any or all of these attempts prove unsuccessful in better coordinating current national 
data collections to provide State-level data on student postsecondary status, other strategies should be 
considered. One would be for NCES to provide leadership : .n developing a common set of questions 
that States might use to design their own surveys to address these issues. For example. NCES could 
sponsor a demonstration project that would provide a model or models that States could use to acquire 
their own data. The NCES longitudinal studies could then provide a national benchmark with which 
to compare State or district findings. 

4. Student Attitudes and Aspirations 

Student attitudes about their education and schooling experiences are not generally thought to 
be education outcomes. Yet, periodically measuring student attitudes toward school and learning, 
self-esteem and confidence, level of satisfaction with their schooling experiences, and future career 
aspirations can provide important indicators of a number of valued education outcomes. Such 
measures could help to address a number of high interest policy issues including the extent to which 
today's students are likely to be "life long learners," their anticipated job performance, and the 
adequacy of our future labor supply in desired fields. Assessing trends over time on student attitudes 
and aspirations can be particularly illuminating, as can comparisons between major demographic 
subgroups. However, while information on student attitudes is already gathered through NAEP and 
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NCES' longitudinal studies, trends over time have not been reported, and comparable State-level data 

are absent. 

While the National Forum believes that our national education data system should ultimately 
include measures of student attitudes and aspirations, we also believe that the creation and reporting 
of such measures should proceed judiciously. Because the regular monitoring and reporting of this 
type of information at the State and national levels has been so uncommon, developmental work in 
this area is especially needed. Such work needs to focus on both issues of the technical adequacy of 
potential attitude/aspiration measures and their perceived usefulness to members of the education 
policy community. 

C. Summary of Recommendations for Improving the Collection and 
Reporting of Student Outcome Statistics 

Policymakers at the national. State, and local levels want to know more about the results of 
the education process. National, regional, and State goals for education are being identified. To 
monitor regress in reaching these goals, it will be necessary to obtain reliable and credible outcome 
measures that are comparable across States. Because of the interest in reducing disparities that exist 
in current levels of education achievement and attainment among important subgroups of the 
population, all outcome measures should be gathered and reported by race/ethnicity and sex. 

Student outcome measures should be gathered in the four sub-domains--student achievement, 
student participation and progression, student status after high school, and student attitudes and 
aspirations. 
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The National Forum makes the following 1 1 recommendations tor improving data collection 
and reporting in the domain of student outcome statistics across the four key sub-domains: 

Student Achievement 

1. Comparable and uniform student achievement measures (using the State National 
Assessment of Educational Progress JState-NAEPl, if proven valid and reliable) should 
provide State-by-State comparisons of knowledge in core content areas (reading, 
writing, mathematics, science, history and geography) in grades 4. 8. and 12 at least 
once every four years. Knowledge in other subject areas such as literature, music, 
art, computer applications, and civics should also be periodically assessed to the extent 
feasible. 

2. Differences in performance among important subgroups of students should be 
examined and reported at the national and State levels. Subgroups should include 
those traditionally associated with sex, race and ethnic origin, economic status, and 
language status. Provision should be made for States, if they wish, to analyze the 
sample of the student achievement study in their States so that comparisons could be 
made among education units by significant subgroups. 

3. Trends in student performance over time should be reported tor all grades and subjects 
in which the achievement data are collected at the national and State levels. However, 
reporting trends over time should not restrict the development and use of new 
assessment forms that tap a broader range of student proficiencies than those typically 
associated with "paper and pencil" tests. 
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4. The Office of Educational Research and Improvement (OERI), including the NAEP 
program, should give priority to research, development, and experimentation with new 
assessment techniques that can provide broader and more sophisticated measures of 
student performance. 

5. State-by-State student achievement measures should include, in each administration, a 
performance assessment eomponent(s). OERl should enter into cooperative research 
and development arrangements with State and local large-scale assessment programs. 

6. Student achievement results should be scaled in a way to allow comparisons with 
international achievement measures such as those from the International Assessment of 
Educational Progress (IAEP) and the International Association for the Evaluation of 
Educational Achievement (1EA). Comparisons with international achievement 
measures should be made on a regular basis in order to monitor progress in meeting 
the recently developed national education goal adopted by the Governors and the 
President, 

7. Information should be collected on courses of study completed at the time of national 
and State student achievement assessments so that links might be made between 
courses/curriculum completed and assessment results. 

8. Discussion should continue into possible linkages of specific features of the NAEP and 
NELS survey instruments as well as better coordination of the two surveys by NCES. 
One possibility is to equate the NEKS achievement instruments to the NAEP items. 
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Student Participation and Progression 

9. NCES, in cooperation with State departments of education, should obtain and 
periodically report comparable State-by-State data on school dropouts and completers 
by race/ethnicity, sex, and other important subgroups. The specific measures 
calculated should include: 

• An annual dropout rate as defined in the SCVS Dropout Field Test or as modified by 
the results of the field test; 

• A synthetic cumulative dropout rate; and 

• A school completion rate incorporating, to the extent feasible, the 
recommendations of the CCSSO School Completion Task Force. 

Student Status After High School 

10. NCES, in cooperation with other Federal agencies and State departments of education, 
should investigate the feasibility of obtaining and periodically reporting comparable 
State-by-State data on the following subjects by race/ethnicity, sex, and other 
important subgroups: 

• The percentage of high school graduates who enroll in different types of 
postsecondary institutions within a year of graduation; 

• The percentage of high school graduates who enter the military within a year 
of graduation; 

• The percentage of high school graduates who enter the civilian labor force 
within a year of their graduation; and 

• The percentage of high school graduates in the civilian labor force who are 
employed/not employed one year after their graduations. 
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Student Attitudes and Aspirations 

1 1 . OERI should fund special studies related to the regular collection and reporting of data 
on student attitudes toward education and schooling and future aspirations. These 
studies should investigate both the technical validity and reliability of potential 
statistics of this type and their perceived usefulness for purposes of education 
policymaking and planning. 
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Improving Our National Education Data System: 
Summary and Conclusions 



After several months of work, and with the support of a cadre of national experts and 
education stakeholders, the National Forum on Education Statistics has developed an initial action 
agenda for improving the usefulness of nationally collected and reported education statistics. This 
agenda is composed of 36 specific data improvement recommendations addressing four key data 
domains-rtiKfeaf and community background statistics, education resource statistics, school process 
statistics, and stu'ent outcome statistics. These recommendations, which appear in Chapters 2-5, are 
restated here: 

For the domain of student and community background statistics, the National Forum 
recommends the following: 

1. Using data extracted from State administrative record systems on the universe of public school 
students, the National Center for Education Statistics (NCES) should annually collect and 
report State- and national-level aggregates on the following student background characteristics: 

• Fall membership counts by race/ethnicity by grade; and 

• Fall membership counts by sex by grade. 

2. NCES should annually report State- and national-aggregate statistics collected by other 
agencies on the following student subgroups: 

• Handicapped students served, by type of handicap; 

• Free lunch participants; and 

• Participants in compensatory, bilingual, and vocational education programs. 
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3. NCES, in cooperation with other Federal and State agencies, should work toward the regular 
collection and reporting of the following State and national student background statistics: 

• Limited-English -proficiency status; 

• Student handicapping conditions by race; 

• Participation in prekindergarten educational programs; 

• Student health status (e.g., nutrition, health-related absenteeism, and drug and alcohol 
use); and 

• Student mobility and migrant status. 

4. The Office of Educational Research and Improvement (OERI) should fund special studies 
investigating the efficacy of using free-lunch data as proxies tor student socioeconomic status 
(SES), and the costs, benefits, and burdens associated with regularly collecting and reporting 
alternative SES measures. These studies should specifically examine issues of validity, 
reliability, and usefulness of free-lunch and alternative measures tor different types of 
reporting and analysis as well as administrative issues related to the collection and reporting of 
such measures. 

5. NCES should develop the capacity to collect and report data on private school student 
background characteristics parallel to those being developed for the universe of public school 
students. Data might come from the NCES Private Scht>ol Survey and the Schools and 
Staffing Survey, and they should be reported as national aggregates and, to the extent feasible, 
as State aggregates. 

6. In reporting measures of educational resources, school processes, and student outcomes from 
its sample and universe surveys, NCES should attempt, to the extent feasible and appropriate. 
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to provide disaggregated data using the following student and community background 
characteristics: 

• Sex; 

• Racial/ethnic group affiliation; 

• Limited-English-proficiency status; 

• Community wealth: and 

• Family income. 

NCES should consider reporting distributional patterns for the following student and 
community background variables in conjunction with particular resource, process, and 
outcome measures: 

• Public/private school enrollment; 

• Student employment status; 

• Measures of family background (e.g.. parents* education, language spoken in the home); 

• Student mobility; and 

• Student handicapping condition. 
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For the domain of education resource statistics, the National Forum recommends the following: 

t. The National Center for Education Statistics (NCES) should collect and report a set of 

national- and State-level education revenue, expenditure, and human resource measures on an 
annual basis, using data hems from the "National Public Education Financial Survey" and the 
Common Core of Data (CCD) Nonfiscal Surveys. 

2. NCES should continue to provide training and technical support to States to "crosswalk" data 
elements specified by the current CCD Financial Survey as well as other assistance necessary 
for meeting the Handbook 2R2 classifications. 

3. NCES and other Federal agencies should investigate the feasibility of developing a State-by- 
Stare statistical measure to adjust education resource data for differences among States and to 
report education resource trends over time in constant dollars. 

4. NCES and other Federal agencies should investigate the feasibility of developing a State-by- 
State statistical measure to adjust salary data for differences among States and to report 
education salary trends over time in constant dollars. 

5. NCES and other Federal agencies should engage in research and development efforts that will 
enable them to make accurate, comparable, and informative international comparisons of U.S. 
education resource commitments with those of other industrialized nations. 

6. NCES should continue to collect and report data from the CCD aggregated to the State level 
on an annual basis. However, the Center should, over time, develop policies and procedures 
for the regular collection and reporting of district-level resource data. In moving toward 
district-level resource collections, NCES should be particularly cognizant of (1) identifying 
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potential reports such data could generate and (2) the capacity of States to provide district- 
level data. 

NCES should expand the annual CCD "Sta»e Administrative Records Survey" to include: (1) 
an average teacher salary measure that takes into account contract, career ladder, and other 
special incentive pay and (2) a teacher salary measure that takes into account degree status and 
experience. 

NCES should make a long-term commitment to establishing a program- and functionally based 
accounting system. This will provide NCES. policy analysts, and other education researchers 
with better information about how education funds are spent and make it possible to relate 
program resources to the specific education needs of students. The particular program levels 
to be collected should be determined after additional study, taking into account the costs and 
burdens associated with the development of comparable definitions of relevant program 
categories across different locales. 

NCES should expand the Federal Government's survey of private schools to include resource 
information. Wherever feasible, the Center should report private-school resource data from 
its surveys on a State-by-State basis. 

NCES should establish, as a long-term objective, the collection of data regarding the status of 
buildings, including the number, age, condition, and facility needs of the Nation's schools. 

NCES should regularly report data on the number and descriptive characteristics (i.e., age. 
sex, race) of instructional, instructional support, and noninstructional staff in the Nation's 
schools. Such data should be reported at the State level to the extent feasible. 
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NCES should establish, as a long-term objective, measures that indicate total dollar 
investments in education personnel. These measures should be specific to different types of 
staff (e.g., teachers, administrators, instructional aides) and include both direct compensation 
expenditures (salaries) and indirect compensation (fringe benefits). 



For the domain of school process statistics, the National Forum recommends the following: 

1 . The National Center for Education Statistics (NCES) should regularly collect and report 
national and comparable State-level data on student enrollment in academic and vocational 
secondary courses by race/ethnicity, sex, and other demographic subgroups as feasible and 
appropriate. To accomplish this, NCES must first develop procedures for ensuring the 
collection of broadly comparable data across States on secondary-school course offerings. 
The Office of Educational Research and Improvement (OERI) should also determine the 
usefulness of coHevting State-level data on time allocated to subjects in the elementary grades 
(such as that currently collected in the Schools and Staffing Survey |SASS|). 

2. NCES should regularly collet and rej>ort data at the national level on broad indicators of 
teacher preparation (e.g., certification status, number of courses taken in teaching area, major 
field, and preservice and inservice development and training experiences) oy specific teaching 
assignment. Trends on these measures should be related directly to changes in the size of the 
teacher work force as well as student enrollment patterns (i.e., teacher supply and demand). 
In addition, NCES should investigate the feasibility of regularly collecting and reporting 
comparable State-by-State statistics using such measures and in reporting on the numbers of 
new teachers certified via "alternative" routes. 

3. NCES should regularly collect and report data at the national level on student "opportunities 
to learn" specific instructional topics. Work should begin on the high-priority subjects 
included in the national education goals (English, mathematics, science, history, and 
geography) and then proceed to other subjects. OERI should develop new measures of the 
depth and breadth of coverage for these topics for possible future collection and reporting at 
the national and State levels. 
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4. NCES should regularly collect and report nationally representative data on the school 
environment including school-level measures of academic emphasis (e.g., curricular offerings 
and enrollments) and decisionmaking practices. To the extent feasible, NCES should relate 
such data to important background characteristics of students attending these schools (e.g. sex, 
race/ethnicity, handicapping condition, socioeconomic status) as well as to key demographic 
characteristics of the larger school community. 

5. In order to measure progress in meeting the national goal of "safe, disciplined, and drug-free 
schools" (goal No. 6 adopted by the Nation's Governors and the President), NCES or other 
Federal agencies should regularly collect and report national- and State-level data on drug and 
alcohol use and violence in the schools, as well as on policies and programs undertaken to 
prevent such occurrences. To develop measures of these, NCES should proceed immediately 
to examine the feasibility of augmenting its current sample surveys (e.g., SASS), mounting 
new surveys (e.g., using the Fast Response Survey System), or working in concert with other 
agencies concerned with these issues (e.g.. Centers for Disease Control, Drug Enforcement 
Agency). To the extent feasible, tnest data should be related to the background characteristics 
of students and their home communities. 

6. OERI should fund special studies to improve the measurement of important school processes 
including academic emphasis, subject specific instructional strategies, depth and breadth of 
content coverage, the use of new technologies in instructional programs (e.g. personal 
computers), and methods of training teachers and assessing their competence. Newly 
developed measures created through such special stvdies may eventually be incorporated into 
future regular national collections and reports. 



112 

ERIC 



For the domain of student outcome statistics, the National Forum recommends the following: 
Student Achievement 

1. Comparable and uniform student achievement measures (using the State National Assessment 
of Educational Progress IState-NAEPJ. if it is proven valid and reliable) should provide State- 
by-State comparisons of knowledge in core content areas (reading, writing, mathematics, 
science, history, and geography) in grades 4. 8. and 12 at least once every four years. 
Knowledge in other subject areas such as literature, music, art, computer applications, and 
civics should also be periodically assessed to the extent feasible. 

2. Differences in performance among important subgroups of students should be examined and 
reported at the national and State levels. Subgroups should include those traditionally 
associated with sex. race and ethnic origin, economic status, and language status. Provision 
should be made for Stales, if they wish, to analyze the sample of the student achievement 
study in their States so that comparisons can be made among education units by significant 
subgroups. 

3. Trends in student performance over time should be reported for all grades and subjects in 
which the achievement data are collected at the national and State levels. However, reporting 
trends over time should not restrict the development and use of new assessment forms tWt tap 
a broader range of student proficiencies than those typically associated with "paper and 
pencil" tests. 

4. The Office of Educational Research and Improvement (OER1), including the NAEP program, 
should give priority to research, development, and experimentation with new assessment 
techniques that can provide broader and more sophisticated measures of student performance. 

113 

153 



5. State-by-State student achievement measures should include, in each administration, a 
performance assessment component(s). OERI should enter into cooperative research and 
development arrangements with State and local large-scale assessment programs. 

6. Student achievement results should be scaled in a way that allows comparisons with 
international achievement measures such as those from the International Assessment of 
Educational Progress (IAEP) and the International Association for the Evaluation of 
Educational Achievement (IEA). Comparisons with international achievement measures 
should be made on a regular basis in order to monitor progress in meeting the recently 
developed national education goal adopted by the Governors and the President. 

7. Information should be collected on courses of study completed at the time of national and 
State student achievement assessments so that links might be made between 
courses/curriculum completed and assessment results. 

8. Discussion should continue into possible linkages of specific features of the NAEP and the 
National Education Longitudinal Study (NELS) survey instruments as well as better 
coordination of the two surveys by the National Center for Education Statistics (NCES). On 
possibility is to equate the NELS achievement instruments to the NAEP items. 

Student Participation and Progression 

9. NCES, in cooperation with State departments of education, should obtain and periodically 

report comparable State-by-State data on school dropouts and completers by race/ethnicity. 

sex, and other important subgroups. The specific measures calculated should include: 

• An annual dropout rate as defined in the NCES Dropout Field Test or as 
modified by the results of the field test; 



• A synthetic cumulative dropout rate; and 



• A school completion rate incorporating, to the extent feasible, the 
recommendations of the Council of Chief State School Officers (CCSSO) 
School Completion Task Force. 

Student Status After High School 

10. NCES, in cooperation with other Federal agencies and State departments of education, should 
investigate the feasibility of obtaining and periodically reporting comparable Stately-State 
data on the following subjects by race/ethnicity, sex, and other important subgroups: 

• The percentage of high school graduates who enroll in different types of 
postsecondary institutions within a year of graduation: 

• The percentage of high school graduates who enter the military within a year 
of graduation; 

• The percentage of high school graduate.- who enter the civilian labor force 
within a year of their graduation: and 

• The percentage of high school graduates in the civilian labor force who are 
employed/not employed one year after their graduations. 

Student Attitudes and Aspirations 

1 1 . OERI should fund special studies related to the regular collection and reporting of data on 
student attitudes toward education and schooling and their future aspirations. These studies 
should investigate both the technical validity and reliability of potential statistics of this type 
and their perceived usefulness for purposes of education policymaking and planning. 
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Expectations and Future Actions 



The recommendations contained in the Guide To Improving the National Education Data 
System provide an ambitious but essential initial blueprint for reform of the national education data 
collection and reporting system. Implementing them will considerably alter the landscape of the 
current national education statistics system. 

Each of the recommendations outlined in the Guide have implications for changing particular 
features of current education data collection and reporting efforts. Figures 3-6 summarize, for each 
domain of the data agenda, the s^ific agencies and national surveys that may be affected by 
implementing each of the Guide's statistical improvement recommendations. For example, 
implementing the first recommendation on student and community background statistics would 
probably require an enhancement of the NCES Common Core of Data surveys and necessitate 
additional State-level data collection responsibilities as well. However, while implementation of the 
second recommendation in this domain is likely to require enhanced Federal-level coordination of 
existing collections, it should not impose any additional responsibilities on State education agencies. 
In fact, a more coordinated Federal collection effort incorporating variables such as free-lunch and 
handicapped-student counts could potentially reduce State administrative data burdens. 

It is important to make four points about the likely data-development implications of the 
recommendations in the Guide. First, it is likely that many of the recommendations can be 
implemented through enhancements or modifications of existing surveys rather than by engaging in 
completely new data collections. This makes the recommendations more feasible and less costly than 
might otherwise be the case. 
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Figurc3 

Potential Data Development Implications of National Forum on Education Statistics Guide Recommendations: 

Student and Community Background Statistics 
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Figure 3 (Cont.) 
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Potential Data Development Implications of National Forum on Education Statistics Guide Recommendations: 
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Figure6 

Potential Data Dcvclqwnent Implications of National Forum on Education 

Student Outcome Statistics 
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Data Implications for: 



Figure 6 (Ccmt) 
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Second, a basic data system infrastructure is being created through the National Cooperative 
Education Statistics System for implementing many of the statistical improvements we contemplate. 
For example, the Cooperative Education Data Collection and Reporting Standards (CEDCARS) 
project, sponsored by the Cooperative System, is in the process of developing basic criteria for 
acceptable education data collection and reporting across a wide range of agencies and types of data. 

Third, there appears to be a rough balance of burdens between the States and the Federal 
Government associated with implementing the recommended improvements. While some of the 
recommendations would probably entail additional State data collection or financial commitment 
(chiefly the Common Core of Data enhancements and State "buy-ins" to NCES sample surveys), 
others represent changes in NCES sample surveys and the types of reports these surveys regularly 
generate, requests for improved Federal data coordination, and expanded research and development 
activity. 

Finally, it is clear that although some recommendations can be implemented relatively 
quickly, others would require considerable time. Nearly all the recommendations in the area of 
school processes, for example, contemplate additional research and development work. Conversely, 
many of the recommendations in the area of improved student achievement measures are already 
consonant with current NCES data-improvement initiatives. 

It is also important to point out again that the National Forum is not an administrative agency. 
We can recommend but not dictate. We fully expect some parts of this agenda to be embraced more 
readily than others by those at the Federal, State, and local levels who would bear the responsibility 
for making the changes needed to implement new national statistical policies. 

What are our expectations for this document? Why have we engaged in this laborious 
consensus-building enterprise to develop recommendations that we have no authority to implement? 
We have created this Guide because we expect it to begin a systematic process of national statistical 
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reform in education. We fully recognize that the ultimate decisions about what data system 
improvements actually occur will depend on many factors, including the cost of proposed 
improvements, who will pay for them, the "fit" between the priorities expressed here and those of 
implementing agencies, and the way in which the proposed new statistics are to be put into operation. 

We focused our discussions about statistical reforms on the issues of unmet and high-priority 
data needs as perceived by members of the education policy community. The Guide is intended to 
provide the necessary substantive context on which to base subsequent discussion of burden, cost, 
timeliness, specific measurement metrics, and the like. 

With this Guide, the National Forum seeks to begin an interchange with data providers that 
recognizes the legitimacy of our concerns about current limits to the useftilness of national education 
statistics. 

More specifically, we expect the tcilowing: 

• That all members and associates of the National Forum commit their constituent 
organizations to investigating with us the possibility of making the statistical system 
improvements necessary to meet the objectives outlined in our data improvement 
recommendations; 

• That this Guide will serve as a basis for subsequent interchanges between members of 
the Forum and implementing agency(ies) at the Federal, State, ami local levels on 
statistical improvement plans and strategies for implementing these recommendations. 
Those parties include relevant operational divisions within NCES, other units with 
statistical collection and reporting responsibilities both within and outside the U.S. 
Department of Education Department, and representatives of State and local education 
agencies. As the sponsoring agency of the National Forum, the National Center for 
Education Statistics should facilitate these interchanges; and 
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• That based on the results of these discussions, the National Forum will develop a 
strategic plan for implementing this report's statistical improvement 
recommendations. The plan will set priorities from among the list of recommended 
improvements, point out the considerations that must be addressed before particular 
recommendations can proceed, and describe the steps to be taken for meeting such 
considerations. 

The National Forum also expects to use this document to guide the work of its other 
committees. For example, the Forum's Technology, Dissemination, and Communications Committee 
is working on improvements in how we go about collecting education data. This Guide will help the 
committee define the variables that should comprise these newly developing data systems. Similarly, 
the Forum's Policies and Procedures Committee will use the document to help shape future Forum 
and Coop * alive System activities, including systems of financial assistance to States so that they may 
develop the capacity to meet the high-priority data improvement needs outlined in the agenda. 

Despite our ambitious hopes for the Guide, we recognize that its impact depends, in large 
part, upon an understanding by the education data community of the broad-based, consensus-building 
approach by which it was derived. We trust that those who develop and implement statistical policies 
in education will take this improvement agenda seriously because they believe, as we do, that creating 
a national education data system based on a spirit of cooperation and consensus building will result in 
higher quality data, superior policymaking, and. ultimately, a more effective and efficient education 
system. 
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